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Preface 


The creation of this book, the seventh in the series, started like the earlier six books. 
After organizing the weeklong Dagstuhl workshop “Visualization and Processing 
of Anisotropy in Imaging, Geometry, and Astronomy” (October 28—November 
2, 2018, Dagstuhl, Germany), we decided to develop a book that would share with 
our readers many of the wonderful research ideas and results reported at the 
workshop or inspired by intriguing discussions at Dagstuhl. After a call for par- 
ticipation, we received contributions from the attendees of the workshop as well as 
from other researchers. 

The central topic of this book is anisotropy. Objects, processes, and phenomena 
that exhibit variations along different directions are omnipresent in science, engi- 
neering, and medicine. The ability to analyze, model, and physically measure such 
anisotropy in each of its application areas is a common theme that resonates with 
mathematicians, engineers, scientists, and medical researchers. Accordingly, we 
divide the thirteen chapters of our book into four coherent parts. 

Our book starts with three chapters that constitute Part I, whose theme is 
Foundations. The first chapter “Variance Measures for Symmetric Positive (Semi-) 
Definite Tensors in Two Dimensions” considers fourth-order tensors that represent 
the covariance of distributions of second-order tensors in two dimensions and have 
the same symmetries as the elasticity tensor. A set of invariants is introduced, which 
guarantees the equivalence of two such fourth-order tensors under coordinate 
transformations. Chapter “Degenerate Curve Bifurcations in 3D Linear Symmetric 
Tensor Fields” investigates fundamental bifurcations in three-dimensional linear 
symmetric tensor fields, with potential applications in the study of time-varying 
tensor fields with multi-scale topological analysis. The last chapter of Part I 
“Continuous Histograms for Anisotropy of 2D Symmetric Piece-Wise Linear 
Tensor Fields” proposes a method to compute iso-contours and continuous his- 
tograms of the anisotropy of 2D tensor fields, using component-wise tensor inter- 
polation. The authors show that the presented technique leads to accurate 
anisotropy histograms. This chapter kindly provides the image on the book cover. 
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The second part of the book contains four chapters on image processing and 
visualization of different types of data. Chapter “Tensor Approximation for 
Multidimensional and Multivariate Data” surveys the topic of data approximation in 
computer graphics and visualization, focusing on tensor approximation 
(TA) methods for multidimensional datasets. In addition, it studies how applying 
TA to vector fields affects important properties such as magnitudes, angles, vor- 
ticity, and divergence. 

In image processing, anisotropic models can improve results by accounting for 
the orientation of image structures, such as object boundaries. Chapter “Fourth- 
Order Anisotropic Diffusion for Inpainting and Image Compression” introduces 
such a model, a fourth-order partial differential equation that involves a fourth-order 
diffusion tensor, generalizing anisotropic edge-enhancing diffusion. This model is 
applied to repair damaged images and to reconstruct full images from a sparse 
subset of pixels, which can serve as a foundation of image compression. 

Chapters “Uncertainty in the DTI Visualization Pipeline” and “Challenges for 
Tractogram Filtering” treat challenges in the processing and visualization of dif- 
fusion magnetic resonance imaging (AMRI) data and are the first chapters on MRI 
techniques to which the rest of the book is devoted. Chapter “Uncertainty in the 
DTI Visualization Pipeline” places emphasis on diffusion tensor imaging (DTI) and 
reviews the origins of uncertainty as well as the techniques developed for modeling 
and visualizing uncertainty in DTI. 

Upon adequate processing, anisotropy information can shed light on important 
open problems. One such example, which has enjoyed a great deal of interest in 
recent years, involves exploiting the anisotropy revealed by diffusion MRI to map 
neural tracts, i.e., the white matter wiring of the brain. Recent studies have shown 
that existing tractography methods suffer from artifactual connections. Chapter 
“Challenges for Tractogram Filtering” reviews methods developed to filter out such 
connections and discusses the associated challenges in this endeavor. 

Part III of this book is devoted to the mathematical modeling of anisotropy, and 
the fitting of such models to measured data. It starts with chapter “Single Encoding 
Diffusion MRI: A Probe to Brain Anisotropy”, which surveys the state of the art in 
modeling diffusion anisotropy within the human brain, as measured by 
traditionally-encoded diffusion MRI featuring one pair of diffusion gradient pulses. 
It provides a broad overview, discussing aspects of neural tissue structure, math- 
ematical representations of the measured signal, and biophysical models and 
challenges in the reliable estimation of their parameters. 

It is well known that MR images are sensitive to ensemble-averaged molecular 
displacements, and a concrete interpretation of diffusion MRI data in terms of 
physical or structural parameters is challenging. Chapter “Conceptual Parallels 
Between Stochastic Geometry and Diffusion-Weighted MRI” sheds light on this 
problem by drawing a parallel of stochastic geometry, a concept that has found 
much success in geology, astronomy, and communications. The authors review 
important results from stochastic geometry and hypothesize how these could be 
useful for a more robust modeling of MRI data. 
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Many specimens of interest comprise a distribution of microscopic, individually 
anisotropic subdomains. An earlier work has shown that diffusion taking place 
within each of such subdomains can be equivalently modeled by envisioning dif- 
fusion to be taking place under a Hookean restoring force. The averaging of ani- 
sotropic signal, either numerically, or naturally due to the presence of randomly 
aligned pores, results in interesting residual features of the diffusion MRI signal that 
are informative of the underlying microstructure. Chapter “Magnetic Resonance 
Assessment of Effective Confinement Anisotropy with Orientationally-Averaged 
Single and Double Diffusion Encoding” investigates these questions for 
diffusion-encoding schemes featuring one as well as two pairs of pulses. 

The last chapter of Part III, chapter “Riemann-DTI Geodesic Tractography 
Revisited”, addresses again the open problem of mapping neural tracts from dif- 
fusion MRI, now from a data modeling point of view. The authors propose a new 
geodesic tractography paradigm by coupling the diffusion tensor to a family of 
Riemannian metrics, governed by control parameters. The optimal controls, and 
corresponding tentative tracts, show a good correspondence with tracts on simu- 
lated data. 

Finally, Part IV comprises two chapters which are primarily concerned with the 
measurement of anisotropy using MRI. Chapter “Magnetic Resonance Imaging of 
T- and Diffusion Anisotropy Using a Tiltable Receive Coil” combines the now 
well-established measurement of diffusion anisotropy with a quantification of 
directionally-dependent transverse relaxation rates, which provide complementary 
information on tissue microstructure. Protocols for reliable measurement of the 
latter are a topic of ongoing research, and this chapter presents results obtained by 
using a tiltable receive coil. 

Finally, chapter “Anisotropy in the Human Placenta in Pregnancies Complicated 
by Fetal Growth Restriction” reports experimental results from measuring diffusion 
anisotropy in the human placenta, comparing pregnancies complicated by fetal 
growth restriction with normal controls. Results suggest that diffusion MRI, 
otherwise primarily used as a neuroimaging technique, can also provide valuable 
information about placental microstructure and could thus help assess placental 
function during pregnancy. 

As you read this book, we hope that you not only enjoy it for its scientific merit 
but also see it perhaps as a source of inspiration. During the review process, the 
COVID-19 pandemic started and is still ongoing at this moment. The people 
involved in the production of this book (authors, reviewers, and editors), many of 
whom are university professors or students, had to work from home due to the need 
for social distancing. In-person meetings have been replaced with online discus- 
sions. For people who are parents of young children, it has been even more difficult 
due to the added tasks of babysitting and/or homeschooling their children. Despite 
all these challenges as well as the constant worry of contracting the virus and the 
stress associated with social distancing, our reviewers strived to honor their com- 
mitment to finish the reviews on time and provided high-quality, constructive 
reviews that have made each one of the chapters stronger. Similarly, our con- 
tributing authors were diligent in the revision of their work, ensuring a timely 
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delivery of this book to the publisher. We wish to express our gratitude to them, for 
not only making this book possible, but also making it possible during this difficult 
period. 

Last but not least, we would like to thank the editors of the Springer book series 
Mathematics and Visualization, as well as Martin Peters and Leonie Kunz 
(Springer, Heidelberg) for their support to publish this book, as well as the board 
and staff of Schloss Dagstuhl, for their excellent support in organizing the 
workshop. Dagstuhl once again created an enjoyable atmosphere for open inter- 
disciplinary exchange between researchers from different fields. Without this 
unique setting, many participants most likely never would have had the opportunity 
to engage with each other’s work. Finally, we would like to thank the Department 
of Mathematics and Computer Science of Eindhoven University of Technology for 
making it possible to publish this book open access. 
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Foundations 


Variance Measures for Symmetric R) 
Positive (Semi-) Definite Tensors in Two TEE 
Dimensions 


Magnus Herberthson, Evren Özarslan, and Carl-Fredrik Westin 


Abstract Calculating the variance of a family of tensors, each represented by a sym- 
metric positive semi-definite second order tensor/matrix, involves the formation of a 
fourth order tensor Rapca. To form this tensor, the tensor product of each second order 
tensor with itself is formed, and these products are then summed, giving the tensor 
Rabca the same symmetry properties as the elasticity tensor in continuum mechanics. 
This tensor has been studied with respect to many properties: representations, invari- 
ants, decomposition, the equivalence problem et cetera. In this paper we focus on the 
two-dimensional case where we give a set of invariants which ensures equivalence 
of two such fourth order tensors Rapca and Rabcd. In terms of components, such an 
equivalence means that components R;;,; of the first tensor will transform into the 
components R; jki Of the second tensor for some change of the coordinate system. 


1 Introduction 


Positive semi-definite second order tensors arise in several applications. For instance, 
in image processing, a structure tensor is computed from greyscale images that cap- 
tures the local orientation of the image intensity variations [10, 17] and is employed 
to address a broad range of challenges. Diffusion tensor magnetic resonance imaging 
(DT-MRP)) [1, 5] characterizes anisotropic water diffusion by enabling the measure- 
ment of the apparent diffusion tensor, which makes it possible to delineate the fibrous 
structure of the tissue. Recent work has shown that diffusion MR measurements of 
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restricted diffusion obscures the fine details of the pore shape under certain exper- 
imental conditions [11], and all remaining features can be encoded accurately by a 
confinement tensor [19]. 

All such second order tensors share the same mathematical properties, namely, 
they are real-valued, symmetric, and positive semi-definite. Moreover, in these dis- 
ciplines, one encounters a collection of such tensors, e.g., at different locations of 
the image. Populations of such tensors have also been key to some studies aiming to 
model the underlying structure of the medium under investigation [8, 12, 18]. 

Irrespective of the particular application, let Rap denote such tensors,' and we 
shall refer to the set of n tensors as {RO} i. Our desire is to find relevant descriptors 
or models of such a family. One relevant statistical measure of this family is the 
(population) variance 


Ves cy Se Be ses iy = A 
` (i) G) ` (i) p(i) 
j (Ri, = Rav) (Roy = Rea) = F Rip Rei — RavRea , 


i=l i=l 


where Rap = 1 Yai RË) is the mean. (For another approach, see e.g., [8]). In this 
paper, we are interested in the first term, i.e., we study the fourth order tensor (skipping 


the normalization) 
n 


Rana = RORY, RÊ =0, a) 


i=1 


where RË) > 0 stands for R a) being positive semi-definite. It is obvious that Rabca 
has the symmetries Rabca = Roacd = Rabac and Rabca = Reda, 1-€-, Rabca has the 
same symmetries as the elasticity tensor [14] from continuum mechanics. The elas- 
ticity tensor is well studied [13], e.g. with respect to classification, decompositions, 
and invariants. In most cases this is done in three dimensions. The same (w.r.t. sym- 
metries) tensor has also been studied in the context of diffusion MR [2]. 

In this paper we will focus on the corresponding tensor Rapca in two dimensions. 
First, there are direct applications in image processing, and secondly, the problems 
posed will be more accessible in two dimensions than in three. In particular we study 
the equivalence problem, namely, we ask the question: given the components R; jx; 
and R; jki Of two such tensors do they represent the same tensor in different coordinate 
systems (see Sects. 2.1.2 and 4)? 


1.1 Outline 


Section 2 contains tensorial matters. We will assume some basic knowledge of ten- 
sors, although some definitions are given for completeness. The notation(s) used is 


lFor the notation of tensors used here, see Sect. 2.1. 
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commented on and in particular the three-dimensional Euclidean vector space Vab) 
is introduced. 

In Sect. 2.1.2, we make some general remarks concerning the tensor Rabca and 
specify the problem we focus on. Section 2.1 is concluded with some remarks on the 
Voigt/Kelvin notation and the corresponding visualisation in R°. 

Section 2.2 gives examples of invariants, especially invariants which are easily 
accessible from Rabca. Also, more general invariant/canonical decompositions of 
Rabea are given. 

In Sect. 3, we discuss how the tensor Rapca can (given a careful choice of basis) 
be expressed in terms of a3 x 3 matrix, and how this matrix is affected by a rotation 
of the coordinate system in the underlying two-dimensional space on which Rapca is 
defined. 

In Sect. 4 we return to the equivalence problem and give the main result of this 
work. In Sect. 4.1.1 we provide a geometric condition for equivalence, while in 
Sect. 4.1.2, we present the equivalence in terms of a 3 x 3 matrix. Both these char- 
acterisations rely on the choice of particular basis elements for the vector spaces 
employed. In Sect. 4.1.3 the same equivalence conditions are given in a form which 
does not assume a particular basis. 


2 Preliminaries 


In this section we clarify the notation and some concepts which we need. Section 2.1 
deals with the (alternatives of) tensor notation and some representations. The equiv- 
alence (and related) problems are also briefly addressed. Section 2.2 accounts for 
some natural invariants, traces and decompositions of Rabca- 

We will assume some familiarity with tensors, but to clarify the view on tensors we 
recall some facts. We start with a (finite dimensional) vector space V with dual V*. A 


tensor of order (p,q) is then a multi-linear mapping V x V--. x Vx V* x +--+ x V* 
a a E 


q p 
— R. Moreover, a (non-degenerate) metric/scalar product g : V x V —> R gives an 


isomorphism from V to V* through v > g(v, -), and it is this isomorphism which 
is used to ‘raise and lower indices’, see below. Indeed, for a fixed v € V, g(v,-)isa 
linear mapping V —> R, i.e., an element of V*. 


2.1 Tensor Notation and Representations 


There is a plethora of notations for tensors. Here, we follow the well-adopted conven- 
tion [16] that early lower case Latin letters (T“;,) refer to the tensor as a geometric 
object, its type being inferred from the indices and their positions (the abstract index 
notation). gq, denotes the metric tensor. When the indices are lower case Latin letters 
from the middle of the alphabet, T“ jk, they refer to components of T“;, in a certain 
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frame. The super-index i denotes a contravariant index while the sub-indices j, k 
are covariant. For instance, a typical vector (tensor of type (1, 0)) will be written 
v! with components vi, while the metric ga, (tensor of type (0, 2)) has components 
gij- At a number of occasions, it will also be useful to express quantities in terms of 
components with respect to orthonormal frames, i.e., Cartesian coordinates. This is 
sometimes referred to as “Cartesian tensors’, and the distinction between contra- and 
covariant indices is obscured. In these situations, it is possible (but not necessary) to 
write all indices as sub-indices, and sometimes the symbol = is used to indicate that 
an equation is only valid in Cartesian coordinates. For example T; = T; jk5 jk instead 
of Tİ = T' jpg? = T'*,. Often this is clear form the context, but we will sometimes 
use = to remind the reader that a Cartesian assumption is made. Here, the Einstein 


summation convention is implied, i.e., repeated indices are to be summed over, so 
n n n 


that for instance Tİ = T' jag’ =T=} T' jpg’ = J. 7'*, if each index 
j=1k=1 k=1 
ranges from 1 to n. We have also used the metric g;; and its inverse g to raise and 
lower indices. For instance, since g;;v' is an element of V*, we write g;jv' = vj. 
We also remind of the notation for symmetrisation. For a two-tensor Tab) = 
5 (Tab + Tha), while more generally for a tensor 7j,4,...a, Of order (0, n) we have 


Tayay--a,) = zl y Tararo -ant 
n 


where the sum is taken over all permutations x of 1, 2, . . . , n. Naturally, this conven- 
tion can also be applied to subsets of indices. For instance, Habe) = 5 (Habe + Haco). 


2.1.1 The Vector Space of Symmetric Two-Tensors 


In any coordinate frame a symmetric tensor Rap (i.e., Rap = Rpa) is represented by 
a symmetric matrix R;; (2 x 2 or 3 x 3 depending on the dimension of the underly- 
ing space). In the two-dimensional case, with the underlying vector space V“ ~ R? 
, this means that Rap lives in a three-dimensional vector space, which we denote 
by Vab). Vab) is equipped with a natural scalar product: < Aab, Bab >= AaB, 
making it into a three-dimensional Euclidean space. Here Aap B® = Aap Bea gt eet 5 
i.e, the contraction of Aq, Bca over the indices a,c and b, d, and the tensor prod- 
uct A,,B.q itself is the tensor of order (0, 4) given by (Agp Bea) vu? wom! = 


(Aab vu? \(Bea wom! ) together with multi-linearity. 
2.1.2 The Tensor Rapca and the Equivalence Problem 
As noted above, Rabca given by (1) has the symmetries Rabca = Riab\cd = Ravied) 


and Raped = Redap, and it is not hard to see that this gives Rabca six degrees of 
freedom in two dimensions. (See also Sect. 2.1.3.) It is also interesting to note that 
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Rabca provides a mapping Vab) > Vab) through 
Rab > Rabed Re, 


and that this mapping is symmetric (due to the symmetry Rabca = Redap). Given 
Rabca there are a number of questions one can ask, e.g., 


e Feasibility—given a tensor Rapca With the correct symmetries, can it be written in 
the form (1)? 

e Canonical decomposition—given Rabca of the form (1), can you write Raped as a 
canonical sum of the form (1), but with a fixed number of terms (cf. eigenvector 
decomposition of symmetric matrices)? 

e Visualisation—since fourth order tensors are a bit involved, how can one visualise 
them in ordinary space? 

e Characterisation/relevant sets of invariants—what invariants are relevant from an 
application point of view? 

e The equivalence problem—in terms of components, how do we know if Rj;x; and 
R; jki represent the same tensor when they are in different coordinate systems? 


We will now focus on the equivalence problem in two dimensions. This problem 
can be formulated as above: given, in terms of components, two tensors (with the 
symmetries we consider) R;;xj and R; jkl, do they represent the same tensor in the 
sense that there i is a coordinate transformation taking the components R;;,; into the 
components R; jk? In other words, does there exist an (invertible) matrix P”; so that 


D m pn o p 
Rijk = Rmnop P iP jP xP? 1? 


This problem can also be formulated when R; jx; and R; jki are expressed in Cartesian 
frames. Then the coordinate transformation must be a rotation, i.e., given by arotation 
matrix Q' ; € SO(2). Hence, the problem of (unitary) equivalence is: Given R; jx; and 
R;jx, both expressed in Cartesian frames, is there a matrix (applying the ‘Cartesian 
convention’) Q;; € SO(2) so that 


Rijki = Rmnop Oni Qnj Qok Qp? 


2.1.3 The Voigt/Kelvin Notation 


Since (in two dimensions) the space Vap) is three-dimensional, one can introduce 
, x fie, 

coordinates, for example Rj; = G z) ~ (>) and use vector algebra on R°. This is 

used in the Voigt notation [15] and the related Kelvin notation [6]. As always, one 


x 
must be careful to specify with respect to which basis in Vap) the coordinates (2) 


sae x 
are taken. For instance, in the correspondence Rj; = (52) ~ G ), the understood 


basis for Vap) (in the understood/induced coordinate system) is {(4 8), (94), (8?)}. 
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Fig. 1 Left: the symmetric matrices e9, e” ; e9 (red) and eS + e9 3 e® + e®) (blue) as vectors 


in R>. The positive semi-definite matrices correspond to vectors which are inside/above the indicated 
cone (including the boundary). Right: the fourth order tensors ce? + 0D ye? + gO ) and (eP 4 


ab T 
ele ey + e8) ) depicted in blue, and DA shown in red are viewed as quadratic forms and 


illustrated as ellipsoids (made a bit ‘fatter’ than they should be for the sake of clarity) 


These elements are orthogonal (viewed as vectors in Vap) ) to each other, but not (all 
of them) of unit length. 

Since the unit matrix plays a special role, we make the following choice. Starting 
with an orthonormal basis {é, ñ} for V, (ie. Ae Le for V“)a a orthonormal 


2 3 
basis for Vian) is {e9}, e9, eS} where gos = (Eto — nano), ef = Jz (Gano + 


Nakb)s e = = Fab + Nano), i.e., in the induced basis we have 


1 /1 0 7 1 /01 A 1 /10 á 
D in aS 2) =a ~ 
ĉij V2 k °) Wa Gij J2 G 5) Y fij /2 (o ') BR 


In this basis, we write an arbitrary element Map € Vab) as Mij = (T ne ), which 


— x 
means that Ma, gets the coordinates M; = V2 ( y Note that Mj; is positive definite 
if z? — x? — y? > 0 and z > 0. In terms of the coordinates of the Voigt notation, the 


tensor Rabca Corresponds to a symmetric mapping R? —> R°, given by a symmetric 
3 x 3 matrix, which also shows that the degrees of freedom for Rapca iS Six. 


2.1.4 Visualization in R? 


Through the Voigt notation, any symmetric two-tensor (in two dimensions) can be 
visualised as a vector in | RS, Using the basis vector en ee (2), we note that ae 


and eD correspond to indefinite quadratic forms, while ef ; ) is positive definite. We 


also see that a + e9 and eD + e9 are positive semi-definite. 
In Fig. 1 (left) these matrices are illustrated as vectors in R*. The set of positive 
semi-definite matrices corresponds to a cone, cf. [4], indicated in blue. When the 


symmetric 2 x 2 matrices are viewed as vectors in R*, the outer product of such 
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a vector with itself gives a symmetric 3 x 3 matrix. Hence we get a positive semi- 
definite quadratic form on R?, which can be illustrated by an (degenerate) ellipsoid in 
. . 1 3 1 3 2 3 2 3 3 
R°. In Fig. 1 (right) (e + ee) +e), (eF +e e2 +e) and ee are 
visualised in this manner. Note that all these quadratic fons correspond to matrices 

which are rank one. (Cf. the ellipsoids in Fig. 2.) 


2.2 Invariants, Traces and Decompositions 


By an invariant, we mean a quantity that can be calculated from measurements, 
and which is independent of the frame/coordinate system with respect to which the 
measurements are performed, despite the fact that components, e.g., Tİ jk themselves 
depend on the coordinate system. It is this property that makes invariants important, 
and typically they are formed via tensor products and contractions, e.g., T’ jx Tig. 
Sometimes, the invariants have a direct geometrical meaning. For instance, for a 
vector v’, the most natural invariant is its squared length v'v;. For a tensor T'; of 
order (1,1) in three dimensions, viewed as a linear mapping R? —> R, the most 
well known invariants are perhaps the trace T’; and the determinant det(T‘ j). The 
modulus of the determinant gives the volume scaling under the mapping given by 
T' ;, while the trace equals the sum of the eigenvalues. If TŻ; represents a rotation 
matrix, then its trace is 1 + 2 cos ¢, where ¢ is the rotation angle. In general, however, 
the interpretation of a given invariant may be obscure. (For an account relevant to 
image processing, see e.g., [9]. A different, but relevant, approach in the field of 
diffusion MRI is found in [20].) 


2.2.1 Natural Traces and Invariants 


From (1), and considering the symmetries of Rabca, two (and only two) natural traces 
arise. For a tensor of order (1, 1), e.g., R;’, it is natural to consider this as an ordinary 
matrix, and consequently use stem letters without any indices at all. To indicate this 


slight deviation from the standard tensor notation, we denote e.g., R;/ by R. Using 
[-] for the trace, so that [R] = Tr(R) = R,“, we then have 


n n 
Tap = Rave’ = Y RQ RO? = Y ROR), (3) 


i=1 i=1 


and 


aye 
Sab = = Rach! = RO RO : (4) 


i=l 
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Hence, in a Cartesian frame, where the index position is unimportant, we have for 
the matrices T = Tij S= Sij 


T= Da [R®], S= D 


To proceed there are two double traces (i.e., contracting Rabca twice): 


n n 
ron SR oS) RR => ROP (5) 
i=l i=l 


and 
n n 


S = Sa? = Rac = Y RÈ RO” = > [ROY]. (6) 


i=l i=l 


In two dimensions, the difference Tap— Sap is proportional to the metric gap. Namely, 


Lemma 1 With Tı, and Sap given by (3) and (A), it holds that (in two dimensions) 


n 
To = S= > det(R) gap. 
i=1 


Proof By linearity, itis enough to prove the statement when n = 1,i.e., when the sum 
has just one term. Raising the second index, and using components, the statement then 
is T;/ — S;/ = det(R“)6;/. Putting RO = A, we see that T; — S;/ = A[A] — A? 
while det(R ())§;/ = det(A)/, and by the Cayley-Hamilton theorem in two dimen- 
sions, A[A] — A? is indeed det(A)/. 


From lemma 1, it follows that T — S = 2}; det(R) > 0. In fact the following 
inequalities hold. 


Lemma 2 With T and S defined as above, it holds that S < T < 2S. IfT = S, all 
tensors RÈ have rank 1. If T = 2S, all tensors RÈ are isotropic, i.e., proportional 
to the metric gap. 


Proof Again, by linearity it is enough to consider one tensor R® = A. In an 


orthonormal frame which diagonalises A, we have A = (5 0) (with a > 0,c > 0, 
a + c > 0). Hence 


San te <a +e +2ac = (a+c)? = T = 2a? + c?) — Gey =35. 


The first inequality becomes equality when ac = 0, i.e., when A has rank one. The 
second inequality becomes equality when a = c, i.e., when A is isotropic. 
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Definition 1 We define the mean rank, rm, by Fm = T/S, with T and S as above. 


Hence, in two dimensions, 1 < rm < 2. 


2.2.2 A Canonical Decomposition 


It is customary [3, 7] to decompose a tensor with the symmetries of Rabca into a sum 
where one term is the completely symmetric part: 


Rabed = Flabed F Wabed> where Havca = Riabed), Wabca = Rabed a. Habed- 


It is also customary to split Habca into a trace-free part and ‘trace part’. We start by 
cons Hap = Hane‘, H = Ha" and then the trace-free part of Hap: Hy = = Hap — 

5H gap SO that Hap = Ĥa +5 SH Zab- ies a ir be made in any 
PaA but the actual epeficien, e.g., 5 + above and ¿ = and 3 = et cetera below 
depend on the underlying dimension.) It is straiehitarward t check that 


Habea = Hated — 8 (ab Hea) + + Hg (abcd) = Havea — 8(ab Hea) — + Hg (abcd) 


is also trace-free. Hence we have the decomposition 


k o ° 3 
Habca = Habca + gab Hea) = + Hg (ab8cd) = Habed + 8 (ab Hea) + 3g (ab8ca)- 


Moreover, due to the symmetry of Rabca, we find that 


Habca = i (Rabca + Racba + Radbc) 


and therefore that 
Wabed = 4 (2Rabea — Racbd — Raabe) (7) 


which implies that Hap = Hane = 4 (Tab + 2Sab) and Wap = Wares = $ (Tab — Sab). 
The degres of freedom, which for Rasca iS six, is distributed, where Rabca ~ 
{Habca, Hav, Wabca}, as 


Rabea ~ {Habeas Hav, Wabca} ~ (Habea, Habs H, Wabca}- 
(6) (2) (3) (1) (2) (2) a) (1) 


For Ha, (or the pair Hap, H) this is clear. The total symmetry of Paved leaves only 
five components (in a basishs Éin, Au, Hiss; Hise. Én. However, the trace- 
free condition Habea g! =0i imposes three conditions. (In an orthonormal frame, 
Hin = -Éni A2 = —Ayin and Ain = — Ë.) That Wapea has only one 
degree of freedom follows from the following lemma. 


Lemma 3 Suppose that Wabca is given by (7), and put Wap = Wage, W = 
Wag”. Then (in two dimensions) 
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Wabed == ¥ (28ab8cd — S8ac8bd — Sad Sbc) 


Proof By linearity, it is enough to consider the case when Rabca = Aap Aca for some 
(symmetric) Aap. In terms of eigenvectors (to A%,) we can write Aab = AXgXp + 
Byayp, where xax = yay! = 1, xay“ = 0. In particular gap = XaXp + Yayp. From 
(7) we then get 


Wabed =} (2Rabed ua Racba ig Radbc) 
=} (2Aab Aca E AagcAba = AadAbc) 
=} (L(A xa Xp + BYaYb)(AXcXa + BYcYa) (8) 


a (AXqX¢ + BYaYc)(AXbXa F Bypya) 
—(axXqxXa + BYaVa)(AXpxXe + BYvyYc)) - 


Expanding the parentheses, the components XaXpXcXa and YaYbYcYa Vanish, leaving 


ap 
y PXaXbVeYa F 2YaYbXcXad — XaXc Vb Yd 
— YaVeXbXd — XaXdYbYe — YaYdXbXc) (9) 


B 
=z (28ab8cd — 8ac8bd — Sad Sbc) ; 


where the last equality can be seen by inserting gap = XaXp + Yayp (for all indices) 
and expanding. Taking one trace, i.e., contracting with g“ gives Wap = 708 9.4, and 


another trace gives W = a which proves the lemma. 


3 Rabcd as a Quadratic Form on RÌ 


Through the orthonormal basis for the space of symmetric two-tensors (in two dimen- 
sions) given by (2), the tensor Rabca viewed as a quadratic form can be represented 
by a3 x 3-matrix. m we will restrict ourselves to an orthonormal basis for Vab), 
namely the basis {e? po e9, a} from Sect. 2.1.3, defined in terms of the orthonor- 
mal basis {§“, 7“} for V°. Thus, given Rapca, We associate the symmetric matrix Mj;, 


where (the choice of an orthonormal basis justifies the mismatch of the indices i, j) 
Mij È R® ge (e, 1 < i, j < 3. 


It is instructive to see how the various derived tensors show up in M;j. In terms of 
the basis (2) it is natural to look at the various parts of Mj; as follows 


m = (HE = (Sz). 00 
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This splitting is natural for reasons which will become apparent in the next sections. 
Note, however, that with this representation it is tempting to consider coordinate 
changes in R?, which is not natural in this case. Rather, of interest is the change of 
basis in V“ and the related induced change of coordinates in the representation (10). 
See Sect. 3.2. 


3.1 Representation of the Canonically Derived Parts of Rabtca 


It is helpful to see how the components of the various tensors Tap, Sab, T, S, Habeds 
Ha», H and W show up as components of Mj;;. As for Hap, e.g., Tap denotes the 
trace-free part of Tap. Immediate is M33: 


i . 1 , 1 
M33 = R” ge (e®)4 = zR edgabg = Tye = z (11) 


Similarly, fori = 1, 2 we have 


. 1 > l p 1 
Mi3 = T gef) g = pT eae = = Fe e®, (12) 
where the last equality follows form the trace-freeness of el and eD. This means 


that the components of Tas (properly rescaled) goes into M;; as the components 
of y (and vy’) in (10). The same holds for Se and Aap, as Sa = Ta by Lemma 1, 
which then implies that also Hap = Te = ae This latter relation follows from the 
trace-free part of the relation Hap = 5 (Tab + 25S,,). Hence 


= Aes 
A|T JEF T 

Mij ara = => , (13) 
T [4T f lir 


> > > 
where 7 = $ = H encodes the two degrees of freedom in Tp = = Sub = = Ĥa. The 


matrix A is decomposed as A = FJ + A where I is the (2 x 2) identity matrix and 


A is trace-free part of A. In particular, [A] = ø. 
To investigate [M;;] = My, + Mz. + M33, i.e., the trace of Mij we note that 


for a general symmetric matrix Rij = (4°) we have R; eD = Rije? = 


ijĉij F 
2, R,;e® = ik When Mj; is constructed from Rapca which is an outer prod- 


uct Rap Rca the trace is given by Mj, + Mx + M33 = (CF £)? + (2 a) + (SS y= 


a? +2b* +c? and from (6) this is S. Together with Taai m o that 
[M] = Mi, + M22 + M33 = S also when Rabca is formed as in (1). Taking trace 
in (13), this gives 


S=0 +iT, i.e, o = S — 
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In addition, the relations below Eq. (7) show that 


H=1(7 +28 T=H+W 
sh t2) ie T so that o = 


3 ies i iH- W. 
W=4(T-85) S=H-35W 


The two degres of freedom in A corresponds to the two degrees of freedom in Habed- 


3.2 The Behaviour of M;; Under a Rotation of the 
Coordinate System in V“ 


The components of M;; are expressed in terms of the orthonormal basis tensors given 


by (2), and these in turn are based on the ON basis {é, n} for V. Putting the basis 
vectors in a row matrix (£ ñ) and the coordinates in a column matrix ($) so that 


a vector U = & Ê +n = (£ ĝ) ( ; ), and considering only orthonormal frames, the 
cos v — sin ’) 


relevant change of basis is given by a rotation matrix O(v) = O, =|. 
g g y QV) = Q, SIAE COs 


i.e., we consider the change of basis 
aa aa a ,\ [cosv —sinv aA 
ED > (Ei) =E (Sey con’) = ENW 
This means that for a vector u = ( â) (*) = (ê ĝ) a the coordinates transform 


Q-Q eo eoe, 


For the components of the basis vectors e ; eo A we find (omitting the factor 


1//2) 
10 = cosy sinv 1 0 cosv—sinv\ _ / cos2v —sin2v 
0-1 —sinv cosv/ \0 —-1 sinv cosy /) \—sin2v —cos2v 
01 = cosy sinv 01 cos v — sin v sin2v cos2v 
10 — sin v cos v 10 sinv cosv cos 2y — sin 2v 
10 cosv sinv 10\ /cosv = sinv\ _ 
01) ? \—sinvcosv/ (01 siny cosv J) | i 


and this means that the components M;; transform as 


. (Alv ~. t AQ» |05 Vv 
Mij = ($ 7) + Why = (S52 2a”), (15) 


(14) 
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But this latter expression is just 


Q},,|0 (4) Q>,|0 
a) (Ha) oT} 


hence we have the following important remark/observation: 


Remark 1 Viewing the matrix Mj; as an ellipsoid in IR, the effect of a rotation by 
an angle v in V^ corresponds to a rotation of the ellipsoid by an angle 2v around the 
z-axis in R? (where the z-axis corresponds to the ‘isotropic direction’ given by gap). 


4 The Equivalence Problem for Rapca 


The equivalence problem for Rabca can be formulated in different ways (for an 
account in three dimensions, we refer to [3]). Given two tensors Rabca and Rabcd, 
both with the symmetries implied by (1), the question whether they are the same or not 
is straightforward as one can compare the components in any basis. However, Rabca 
and Rated could live in different (but isomorphic) vector spaces, e.g. two tangent 
spaces at different points, and the concept of equality becomes less clear. On the 
other hand, in terms of components R;jzı and R; jkl, One could ask whether there is a 
change of coordinates which takes one set of components into the other. If so, one 
can find a (invertible) matrix Pi j so that 


m pn po p 
Rijki = RmnopP ;P jP cP? ), 


and the tensors are then said to be equivalent. As already mentioned, it is convenient 
to restrict the coordinate systems to orthonormal coordinates. This means that two 
different coordinate systems differ only by their orientation, i.e., the change of coor- 
dinates are given by a rotation matrix Q € SO(2). Under the ’Cartesian convention’ 
that all indices are written as subscripts, Rabca and Rabcd are equivalent if there is a 
matrix Q € SO(2) so that (their Cartesian components satisfy) 


Rijki = Rmnop Omi Onj Ook Opi: 


4.1 Different Ways to Characterize the Equivalence of Rabca 
and Rabed 


In this section, we will discuss three ways to determine whether two tensors Raped 
and Rapca are equivalent or not. In Sects. 4.1.1 and 4.1.2 we present two such methods 
briefly, while Sect. 4.1.3, which is more complete, contains the main result of this 
work. 
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Fig. 2 Three identical (truncated) ellipsoids in R? with different orientations. The two leftmost 
ellipsoids can be carried over to each other through a rotation around the (vertical in the figure) 
z-axis, which implies that they represent the same tensor Rabca (up to the meaning discussed). The 
right ellipsoid, despite identical eigenvalues with the two others, represent a different tensor since 
the rotation which carries this ellipsoid to any of the others is not around the z-axis 


As mentioned in Sect. 1.1, the results of Sects. 4.1.1 and 4.1.2, which may be 
used in their own rights, rely on particular choices of basis matrices for Vab). The 
formulation in Sect. 4.1.3 on the other hand, is expressed in the components of Raped 
(in any coordinate system) directly. 


4.1.1 Orientation of the Ellipsoid in Rê 


A necessary condition for Rabca and R, bca to be equivalent is that their corresponding 
3 x 3-matrices Mj; and M; j have the same eigenvalues. On the other hand, this is 
not sufficient since the representation in R? should reflect the freedom in rotating 
the coordinate system in V“ ~ R?. With the coordinates adopted, this corresponds 
to a rotation of the associated ellipsoid around the z-axis in R* (see Remark 1 in 
Sect. 3.2). This is illustrated in Fig. 2 where three ellipsoids, all representing positive 
definite symmetric mappings having identical eigenvalues, are shown. The two first 
ellipsoids can be rotated into each other by a rotation around the z-axis. This implies 
that the corresponding tensors Rabca and Rabed are equivalent. The third ellipsoid 
can also be rotated into the two others, but these rotations are around directions other 
than the z-axis, which means that this ellipsoid represents a different tensor. 

In the generic case, with all eigenvalues different, it is easy to test whether two 
different ellipsoids can be transfered into each other through a rotation around the 
z-axis. This will be the case if the corresponding eigenvectors (of M;; and Mi j) have 
the same angle with the z-axis. Hence it is just a matter of checking the z-components 
of the three normalized eigenvectors and see if they are equal up to sign. 


4.1.2 Components in a Canonical Coordinate System 
In a sense, this is the most straightforward method. In a coordinate system which 


respects e as the z-axis in Vab) ~ R?, two tensors Rabea and Rabed are equivalent 
if there is a rotation matrix (in two dimensions) Q such that 
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> È 
Alt ‘AQ\|O'T 
o 1 = 2 c 2 ~ ` (16) 
T'|3T T'O| 3T 


~ 2 
Hence, equivalence can be easily tested by first checking that T = T and that || T || = 
S 


|I F ||. If mets is the case, (and if || Î || > 0) one determines the rotation matrix Q 


which gives 7 = = me F, and equivalence is then determined by if A = Q' AQ or 
= 


not. If || Î I= || T || = 0, the equivalence of A and A can be determined directly, 
i.e., by checking whether [A] = [A] and [A?] = [A2] or not. 


4.1.3 Equivalence Through (algebraic) Invariants of Rabca 


If a solution is found, this is perhaps the most satisfactory way to establish equiva- 
lence, in particular if the invariants are constructed by simple algebraic operations 
only. (For instance, to asymmetric 3 x 3-matrix A one can take the three eigenvalues 
as invariants or else for instance the traces of A, A? and A>. The former set requires 
some calculations, but the latter is immediate.) 

Examples of invariants are T = Rabca g g“ , S = Radea gy” ght and the invariants 
H = Hap g , W = Wap g”. To produce the invariants, we use the tensor Rapca and 
the metric gap. However, if we regard V^ ~ R? as oriented, so that the orthonormal 
basis {é, n} for V“ also is oriented, then invariants can also be formed in another way. 
Namely, since the space of symmetric 2 x 2 matrices is 3-dimensional, and since the 
metric gap singles out a 1-dimensional subspace, it also determines a 2-dimensional 
subspace L; all elements orthogonal to gap. This subspace is the set of all symmetric 
2 x 2 matrices which are also trace-free. L can be given an orientation through an 
area form, which in turn inherits the orientation from V^. 

In general, with right-handed Cartesian coordinates x , x, the area form € is 
given by € = dx! A dx? where (w A Lab = Wally — bha. With the orthonormal 
basis {E, n} (for V° ) also right handed, we define, cf. (2), 


1 


1 pa a A A 2 = A A = 
o = EnEn — hae), em = Jp Êa + iako). (17) 


The area form on L is then € ~ e A e®, or 
De 2 
e ~ Eabod = C56 ed — Ent Eed (18) 


It is not hard to see that this definition is independent of the orientation of 46, ny}. 
We observe that 2Eabcd = ÊE, - hatin Ê. fa + êa) — ait + PAG êa- 
7.04). By replacing é by ô = cos vÊ + sinvĝ and 7 by f= — sin v Ê + cos v Hf, 
i.e., a rotated orthonormal basis, it is straightforward to check that 


18 M. Herberthson et al. 
(@a@p — Aa Rp) (Ôcha + h.@a) — (@afly + Aab) Âa — ft hea) (19) 
=(E,€, — Nay) E cda + 18a) — Eatty + 108) (E064 — Nea) 


so that Eabca is well defined. We recollect that area form Eapca is defined, through 
the induced metric, on the plane L (which in turn is also defined through the metric 
Zab) and the orientation on V“. Hence Eabca can be used when forming invariants. 
We will now state the result of this work, namely the existence of six invariants 

which can be used to investigate equivalence of two tensors Rabca and Rabca. We 
start by defining 

S =Raveas g" 

T =Rapeag g 

Jo =Rapca R” 

Jı =T” Ta 

J2 =Ravc TPT“ 


Jy =T” Rapca E Top. 


(20) 


where Eabca is defined by (17) and (18). Similarly, we define S, T, Jo, Ji, Ja and 
Ts as the corresponding invariants formed from Rabed. We make the remark that for 
most of these invariants, their immediate interpretations still remain to be found. 
Rather, their value lie in the fact that they form a set which can be used to establish 
the equivalence in Theorem 1 below. On the other hand, some interpretations are 
possible. In particular, the quotient T/S (see Definition 1) lies in the interval [1, 2] 
and has the meaning given by Lemma 2. 


Theorem 1 Suppose that Rabca = ya | RÖ RË, with RÈ > 0 and that Rijk are 
the components of Rabed in some basis. Suppose also that Rage = Sa RORO, 
with RO > > O and that Riju are the e components of Rapca i in some, possibly unrelated, 
basis. If (and only if) S = S, T= T, Jo = Jo, J= F, Jn = h, J = B, then there 


is a transformation matrix P' ; such that 


Rijki = Rmnop P”: P”; P°“P? 1. 
Proof Since the invariants are defined without reference to any basis, it is sufficient 


to consider the components expressed in an orthonormal frame, and in that case we 
must prove the existence of a rotation matrix Q € SO(2) so that 


Rijki = Rinnop Omi Onj Ook Qp. 


Since 
( 
Rabed = Mij ee), (21) 


we can consider the invariants formed from the components of 


Variance Measures for Symmetric Positive (Semi-) Definite Tensors in Two Dimensions 19 


Ala s Alu 
Mij = Ga) and Mij = ( =) (22) 


A=05,AQn, u= Qm, C=c. (23) 
We make the ansatz 
Sa. b |x 4 +a b |X 
M= , Mj=|_ 6 3ļ-a|7 (24) 
Y Cc 


Through (21) it is straightforward to see that 


S=0 +c, T=2c, Jo = 2(a? + b?) +c? + 0o?/2 + U(x? + y?), 
Jy = 2c? + x? + y3) 


soif S = Š, T = T, Jo = Jo, J) = Ji, it follows thato =F,c=Gae+hP =e + 
b? and x? + y? = X + F°. Since the isotropic part of A, i.e., 5/ is unaffected by 
a rotation of the coordinate system, we consider the traceless parts A= (3 + ), 


RB ~~ oO 
A= E A and the task is to assert a rotation matrix Q such that 


G)-¢G2)2 G)-2G) 


if also Jp = J, J = Jz: Again itis straightforward to calculate the remaining invari- 
ants, and we find 


Jı = 4bxy + 2a(x? — y?) +2 + (4c +0o)(x? + y?) 
J = 4axy — 2b(x? — y?) . 


and similarly for Jo, Jz. Hence, (since o = o, c =C) 


a? +b = X + D 

x? + y? =7 + y 
2bxy + a(x? — y?) = 2037 + G(X? — 9>) 
2axy — b(x? — y*) = 2039 — BG? — 7). 


(25) 


Suppose first that x? + y? > 0. The equality x? + y? = X? + 9° then guarantees the 
existence of the rotation matrix Q which is determined via the relation (5) = Q'(%). 
This can also be expressed as Q{ (3) = 05(3) for some rotation matrices Q1, Q2, 
where Q = Q201. We now choose the rotation matrix Q; so that in the untilded 
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coordinates, y = 0. Similarly we choose Q% so that for the tilded coordinates, we 
get a frame where Y = 0. The equalities between the invariants in (25) then become 


+= P+ 


L=? 
ax? = ax? 
—bx? = -bX , 


so that a = Ẹ, b = b. This proves the theorem when x? + y? > 0. When x? + y? = 
P+F =0,ie.,x = y = X = Y = 0, the remaining equality a? + b? = @ + D? is 
sufficient since we can again choose frames in which b = b=Oanda>0,a>0. 
It then follows that a = a. 


5 Discussion 


In this work, we started with a family of symmetric positive (semi-)definite tensors 
in two dimensions and considered its variance. This lead us to a fourth order tensor 
Rabca With the same symmetries as the elasticity tensor in continuum mechanics. 
After listing a number of possible issues to address, we focused on the equivalence 
problem. Namely, given the components of two such tensors Rabca and Rees. how 
can one determine if they represent the same tensor (but in different coordinate 
systems) or not? In Sect. 4, we saw that this could be investigated in different ways. 
The result of Theorem | is most satisfactory in the sense that it is expressible in terms 
of the components of the fourth order tensors directly. 

There are two natural extensions and/or ways to continue this work. The first is to 
apply the result to realistic families of e.g., diffusion tensors in two dimensions. The 
objective is then, apart from establishing possible equivalences, to investigate the 
geometric meaning of the invariants. The other natural continuation is to investigate 
the corresponding problem in three dimensions. The degrees of freedom of Rabca will 
then increase from 6 to 21, leaving us with a substantially harder, but also perhaps 
more interesting, problem. 
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Degenerate Curve Bifurcations in 3D A) 
Linear Symmetric Tensor Fields giet 


Yue Zhang, Hongyu Nie, and Eugene Zhang 


Abstract 3D symmetric tensor fields have a wide range of applications in medicine, 
science, and engineering. The topology of tensor fields can provide key insight into 
their structures. In this paper we study the number of possible topological bifur- 
cations in 3D linear tensor fields. Using the linearity/planarity classification and 
wedge/trisector classification, we explore four types of bifurcations that can change 
the number and connectivity in the degenerate curves as well as the number and 
location of transition points on these degenerate curves. This leads to four types of 
bifurcations among nine scenarios of 3D linear tensor fields. 


1 Introduction 


Tensor field visualization is an important topic in visualization, with many applica- 
tions in medical imaging, solid and fluid mechanics, material science, earthquake 
engineering, and computer graphics. 

Recent advances on tensor field visualization focus in topology-driven analy- 
sis and visualization of 3D symmetric tensor fields. Degenerate curves are one of 
the most fundamental topological features in a tensor field, and much research has 
focused on the understanding and efficient extraction of degenerate curves from 
piecewise linear tensor fields defined on a tetrahedral mesh [6-8]. 
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In the book chapter, we focus on a problem that has received relatively little 
attention: bifurcations in tensor field topology. To make our investigation effective 
with potential application to real datasets, we focus on 3D linear tensor fields. We 
explore all the theoretically possible bifurcations. Moreover, we have conducted 
experiment to verify whether these bifurcations can occur. 

The rest of the paper is structured as follows. Section 2 reviews past research 
in topology-driven analysis of symmetric tensor fields. In Sect. 3 we review relevant 
mathematical background and results on tensor fields. In Sect. 4 we report the findings 
of our exploration before concluding in Sect. 5. 


2 Previous Work 


Much research exists on 2D and 3D symmetric tensor fields, and we refer the readers 
to the survey by Kratz et al. [4] and Zhang et al. [11] for a more comprehensive 
review. In this book chapter we only refer to the research that is most relevant. 

Delmarcelle and Hesselink [1] introduce the notion of degenerate points for 
2D symmetric tensors, where eigenvector directions are not well-defined. Zhang 
et al. [12] explore the physical meanings of degenerate points in the stress tensor and 
strain tensor from continuum mechanics. 

Hesselink et al. later extend this work to 3D symmetric tensor fields [3] and study 
the degeneracies in such fields. Zheng and Pang [16] point out that triple degen- 
eracies are structurally unstable features. That is, an arbitrarily small perturbation 
to the field will remove such degenerate points. Zheng and Pang further show that 
double degeneracies, i.e., only two equal eigenvalues, form lines in the domain. In 
this work and subsequent research [18], they provide a number of degenerate curve 
extraction methods based on the analysis of the discriminant function of the tensor 
field. Furthermore, Zheng et al. [17] point out that near degenerate curves the tensor 
field exhibits 2D degenerate patterns and define separating surfaces which are exten- 
sions of separatrices from 2D symmetric tensor field topology. Tricoche et al. [9] 
convert the problem of extracting degenerate curves in a 3D tensor field to that of 
finding the ridge and valley lines of an invariant of the tensor field, thus leading to a 
more robust extraction algorithm. More recently, Palacios et al. [6] extract degenerate 
curves using an algorithm for algebraic surface extraction method called A-patches. 
Palacios et al. [5] introduce a number of topological editing operations with which 
a 3D tensor field can be edited for graphics applications. 

Zhang et al. [13] describe a number of important properties of 3D linear tensor 
fields. They [15] show that in a 3D linear tensor field, there are at least two and at 
most four degenerate curves. Roy et al. [8] develop a parameterization with which all 
degenerate points in a 3D piecewise linear tensor field can be extracted efficiently and 
at any given accuracy. Zhang et al. [14] show that there are at most eight transition 
points in a 3D linear tensor field. 
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3 Background on Tensors and Tensor Fields 


In this section we review the most relevant background on 2D and 3D symmetric 
tensors and tensor fields [14]. 


3.1 Tensors 


A K-dimensional (symmetric) tensor T has K real-valued eigenvalues: 4, > à2 > 
... > Ax. The largest and smallest eigenvalues are referred to as the major eigenvalue 
and minor eigenvalue, respectively. When K = 3, the middle eigenvalue is referred 
to as the medium eigenvalue. An eigenvector belonging to the major eigenvalue is 
referred to as a major eigenvector. Medium and minor eigenvectors can be defined 
similarly. Eigenvectors belonging to different eigenvalues are mutually perpendicu- 
lar. 

The trace of a tensor T = (T;;) istrace(T) = Sa 4;. T can be uniquely decom- 
posed as D + A where D = trace] (I is the K-dimensional identity matrix) and 
A = T — D. The deviator A is a traceless tensor, i.e., trace(A) = 0. Note that T and 
A have the same set of eigenvectors. Consequently, the anisotropy in a tensor field 
can be defined in terms of its deviator tensor field. Another nice property of the set of 
traceless tensors is that it is closed under matrix addition and scalar multiplication, 
making it a linear subspace of the set of tensors. 


The magnitude of a tensor T is ||T|| = | je T7 = JZE ?, while the 


determinant is |T| = Tt, Ài. 

A tensor is degenerate when there are repeating eigenvalues. In this case, there 
exists at least one eigenvalue whose corresponding eigenvectors form a higher- 
dimensional space than a line. When K = 2 a degenerate tensor must be a multiple 


of the identity matrix. 


3.2 Tensor Field Topology 


We now review tensor fields, which are tensor-valued functions over some domain 
2 c RË. A tensor field can be thought of as K eigenvector fields, corresponding 
to the K eigenvalues. A hyperstreamline with respect to an eigenvector field e; (p) 
is a 3D curve that is tangent to e; everywhere along its path. Two hyperstreamlines 
belonging to two different eigenvalues can only intersect at the right angle, since 
eigenvectors belonging to different eigenvalues must be mutually perpendicular. 
Hyperstreamlines are usually curves. However, they can occasionally consist of 
only one point, where there is more than one choice of lines that correspond to 
the eigenvector field. This is precisely where the tensor field is degenerate. A point 
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Fig. 1 A wedge (left) and a trisector (right) 


Po € Q is a degenerate point if T(po) is degenerate. One important topological 
feature of a tensor field consists of its degenerate points. 

In 2D, the set of degenerate points of a tensor field consists of isolated points 
under numerically stable configurations, when the topology does not change given 
sufficiently small perturbation in the tensor field. An isolated degenerate point can 
be measured by its tensor index [10], defined in terms of the winding number of 
one of the eigenvector fields on a loop surrounding the degenerate point. The most 
fundamental types of degenerate points are wedges and trisectors, with a tensor 
index of 5 and — F, respectively. Let LT p, (p) be the local linearization of T(p) at a 


degenerate point po = enl i.e., 
0 


_ (41% — xo) + biy — yo) anl — xo) + bi — yo) 
PEP) = ee — xo) + bi2(y — yo) anx — xo) + b2 (y — a) ) 


Ge | 
bu=b2 h 

2 12 
Po is a wedge when ô > 0 and a trisector when ô < 0. When ô = 0, po is a higher- 
order degenerate point. A major separatrix is a hyperstreamline emanating from a 
degenerate point following the major eigenvector field. A minor separatrix is defined 
similarly. 

The total tensor index of a continuous tensor field over a two-dimensional mani- 
fold is equal to the Euler characteristic of the underlying manifold. Consequently, it 
is not possible to remove one degenerate point. Instead, a pair of degenerate points 
with opposing tensor indexes (a wedge and trisector pair) must be removed simul- 
taneously [10]. Figure 1 shows a wedge pattern (left) and a trisector pattern (right), 
respectively. 


Then 6 = is invariant under the change of basis [2]. Moreover, 
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In 3D, a degenerate point can have either two or three eigenvalues being the same, 
the latter of which is structurally unstable. Structurally stable degenerate points have 
two eigenvalues being the same. These eigenvalues are the repeating eigenvalues, 
while the third eigenvalue is the dominant eigenvalue. When the dominant eigenvalue 
is the major eigenvalue, the degenerate point is a linear degenerate point (L). On the 
other hand, when the dominant eigenvalue is the the minor eigenvalue, the degenerate 
point is a planar degenerate point (P). We refer to this linear/planar classification 
of a degenerate point as its L/P classification. Along a degenerate curve, its L/P 
classification does not change. 

A degenerate point can also be classified in a different way, by projecting the 
tensor field onto the plane passing through the degenerate point and perpendicular to 
its dominant eigenvector. The degenerate point is also a degenerate point in the 2D 
projected tensor field. Consequently, the original 3D degenerate point is classified as 
either a wedge (W) or atrisector (T) based on the W/T type of the 2D degenerate point. 
Note that the projection of the 3D tensor field onto other planes not perpendicular 
to the dominant eigenvector does not necessarily have the same W/T classification. 
Along a degenerate curve, the W/T type can change, separated by degenerate points 
that are neither wedges nor trisectors. Such points are transition points. 

Figure 2 illustrates these concepts with one example tensor field. The combination 
of the L/P and W/T classifications of a degenerate point leads to four combinations: 
LW (green), LT (blue), PW (yellow), and PT (red). Note that along a degenerate curve, 
the L/P classification is constant while the W/T classification can change. Therefore, 
a degenerate curve can consist of either green and blue segments, or yellow and 
red segments, but not other combinations of colors. Linear transition points appear 
between green and blue segments, while planar transition points separate yellow and 
red segments. 

The projected tensor fields onto the plane of repeating eigenvalues are shown along 
the degenerate curve. Notice that at a W type point, the projected pattern shows a 
wedge, while at a T type point, the projected pattern shows a trisector. At a transition 
point, the projected pattern shows neither a wedge pattern nor a trisector pattern. 


3.3 3D Linear Tensor Fields 


A 3D linear tensor field is a 3D symmetric tensor field whose tensor entries are linear 
functions of the XYZ coordinates of the points in the domain. It has the following 
form: 


T(x, y,2) = To + xT, + YT; + 2T, (2) 
where To, T}, Ty, and T, are 3D symmetric tensors. A 3D linear tensor field has 


all the aforementioned properties of a general tensor field. However, there are some 
important properties specific to 3D linear tensor fields. 


28 Y. Zhang et al. 


D 


oo 


Fig. 2 Along a degenerate curve, the projection of the tensor field onto the repeating planes can 
exhibit 2D degenerate patterns such as a trisector (a) or a wedge (c). Between segments of wedges 
(yellow) and trisectors (red), transition points can appear (b) 


First, there are either two or four degenerate curves in a 3D linear tensor field [15]. 
Half of these curves are L, and the other half are P. 

Second, there are either zero, two, four, six, or eight transition points in a 3D 
linear tensor field [14]. 

As pointed out [8], the set of degenerate points in a 3D linear tensor field (including 
their asymptotic limits at infinity) is homeomorphic to a topological circle (a loop). 
Figure 3 illustrates this bijective map. The loop is situated on the unit sphere where 
the location of a degenerate point on the loop corresponds to its unit dominant 
eigenvector. Since eigenvectors have a sign ambiguity, there are two such loops. 
Every degenerate point corresponds to exactly one pair of antipodal points on the 
sphere. While L/P cannot change along a degenerate curve, such a switch can occur 
at infinity, which two degenerate curves approach in opposite directions. The number 
of such switch points, called oo points, is the same as the number degenerate curves 
in the field. Note that oo points are neither L nor P. On the other hand, they can be 
either W or T. This leads to nine scenarios [14]: (1) two WW curves, (2) two WT 
curves, (3) two TT curves, (4) four WW curves, (5) two WW curves and two WT 
curves, (6) one WW curve, two WT curves, and one TT curve, (7) four WT curves, 
(8) two WT curves and two TT curves, and (9) four TT curves. 

Along a WW or TT curve, there must be an even number of transition points, 
while on a WT curve there must exist an odd number of transition points. 

The color of the sphere represents the sign of the discriminant of the projected 
tensor field onto the plane whose normal is the displacement vector of the point on the 
sphere from the center of the sphere. The color is cyan if the discriminant is positive 
(wedge type) and magenta if the discriminant is negative (trisector type). Note the 
W/T classification of a degenerate point must match the sign of the discriminant on 
the sphere. That is, green and yellow segments must appear in the cyan region, while 
blue and red segments must appear in the magenta region. A transition point must 
appear on the boundary between cyan and magenta regions. 
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Xe 


Fig. 3 This figure shows the parameterization of all degenerate points in a 3D linear tensor field 
by a topological circle. In the left are the degenerate curves of a 3D linear tensor field, and in 
the right is the topological circle superimposed on the unit sphere. Between the yellow and green 
segments are oo points. In addition, transition points occur either between yellow and red segments, 
or between green and blue segments. Moreover, we color a point in the sphere (right: representing 
a unit vector) cyan if the tensor projected onto the plane perpendicular to this vector has a wedge 
degenerate point. On the other hand, the point is colored magenta if the projected tensor field has 
trisector degenerate point. Notice that the transition points on the degenerate curves (between blue 
and green segments and between yellow and red segments) are precisely on the boundary between 
the cyan and magenta regions 


4 Bifurcations 


We now describe our findings of bifurcations in 3D linear tensor fields. 

First, there can be either two or four degenerate curves. This means that some 
bifurcations can change the number of degenerate curves in the field. We refer to 
those bifurcations which reduce the number of degenerate curves from four to two 
as degenerate curve removal and those increase the number from two to four as 
degenerate curve generation. 

Second, even when the number of degenerate curves does not change, the way 
the oo points are connected can change. We refer to this as degenerate curve recon- 
nection, which is consistent with the topological editing operation of the same name 
from [5]. Note that two degenerate curves of opposite L/P types cannot be reconnected 
as this would generate a degenerate curve with changing L/P type. Consequently, 
when there are only two degenerate curves in the field, they cannot be connected. 
Degenerate curve reconnection can only occur for four degenerate curves. 

Third, the number of transition points in the tensor field can change. Since this 
number must be even, we refer to such bifurcations as either transition point pair 
cancellation if the number is decreased by two or transition point pair generation if 
the number is increased by two. 

Finally, even when the number of transition point does not change in a field, it 
is possible that some transition point has been moved from one degenerate curve to 
another. We refer to such a bifurcation as a transition point relocation. 

Below we consider these bifurcations in the context of the nine scenarios described 
earlier. 
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4.1 Degenerate Curve Removal and Generation 


Note that a degenerate curve generation bifurcation must be the inverse of a degen- 
erate curve removal bifurcation, and vice versa. Consequently, we consider them 
together in this section. 

Figure 4 lists all theoretically possible degenerate curve generation bifurcations 
and their inverse bifurcations. There is a total of nine scenarios, each of which is 
represented as an ellipse (topological disk) with the oo points marked along with 
their W/T types. A degenerate curve removal bifurcation must take a scenario with 
four degenerate curves (four oo points on the ellipse) to one with two degenerate 
curves (two oo points). Conversely, a degenerate curve generation bifurcation must 
take a scenario with two degenerate curves to one with four degenerate curves. 
In addition, when two degenerate curves are removed, two adjacent oo points are 
removed and two oo points remain. Each of the oo point is written as either W or T 
(called a symbol on the ellipse). This means that not any four-symbolled ellipse can 
connect with any two-symbolled ellipse. For example, while the box with four W’s 
can be mapped to either one with two W’s or one with one W and one T, it can not be 
connected with one with two T’s. Similarly, the box with one W and three T’s cannot 
be mapped to one with two W’s. Furthermore, when two symbols are removed, they 
need to be adjacent symbols. Consequently, it is not possible to connect the ellipse 
with two W’s and two T’s that are interleaved to one with two W’s or the one with 
two T’s. 

These constraints give rise to a total of ten theoretically possible bifurcations, 
each of which is given as an edge in Fig. 4. The following figures provide examples 
of degenerate curve removal bifurcations. 

In Fig. 5 (left), there are initially four TT curves, two of which are linear (blue) 
and two planar (red). After the bifurcation (right), the two linear degenerate curves 
become connected by a linear wedge segment (green), which replaces the lost planar 


Ww WwW Ww Ww w T T T T T 
C> 
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Fig. 4 This diagram shows all the theoretically possible degenerate curve removal and generation 
bifurcations in a 3D linear tensor field 
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Fig. 5 This figure shows a degenerate curve removal bifurcation from the case of four TT curves 
(left) to two TT curves (right). The top-middle shows the non-repeating eigenvector manifold for 
the left, while the bottom-middle shows the non-repeating eigenvector manifold for the right. The 
inverse bifurcation, a degenerate curve generation exists, which swaps the before and after scenarios. 
Note that in these bifurcations the number of segments do not change after the bifurcation 


Fig. 6 Another degenerate curve removal bifurcation takes the case of two WW curves and two 


WT curves (left) to two WW curves (right). Note that the number of segments is decreased by two 
after the bifurcation, i.e. two transition points are lost as a result of the bifurcation 


TT curve (red). Furthermore, two transition points appear on the newly connected 
linear degenerate curve. 

The middle column compares the two fields using their parameterization: (upper- 
middle) corresponding to the field in the left, and (lower-middle) corresponding to 
the tensor field in the right. Through the bifurcation, two infinity points are replaced 
by two transition points, resulting in the switch of a red segment (middle-top) to a 
green segment (middle-bottom), which leads to the loss of two degenerate curves 
and the creation of two transition points. 

Figure 6 provide an additional example of such types of bifurcations. Note that 
in this example, the loss of two degenerate curves also results in the loss of two 
transition points, i.e. the loss of two colored segments. This is a different way of 
achieving degenerate curve removal than the example bifurcation in Fig. 5, in which 
the number of segments does not change. 
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4.2 Degenerate Curve Reconnection 


Degenerate curve reconnection bifurcations do not change the number of degenerate 
curves. Instead, they change how the oo points are connected. As mentioned earlier, 
two degenerate curves can be connected only if they have the same L/P type. When 
there are only two degenerate curves, one is L and the other P, which cannot be con- 
nected after reconnection. Consequently, degenerate curve reconnection bifurcations 
only occur for the case of four degenerate curves. 

There is a total six scenarios with four degenerate curves, as shown in Fig. 7. Since 
degenerate curve reconnection bifurcations do not change the number and type of oo 
points, there is at least one theoretically possible bifurcation that connects a scenario 
to itself. Furthermore, the two scenarios with two W’s and two T’s are connected 
through reconnection bifurcations. 

However, it is worth noting that when reconnection occurs, the oo points on 
the two degenerate curves to be reconnected cannot be arbitrarily connected after 
bifurcation. For an example, consider the case of two W’s and two T’s that appear 
on the ellipse in an alternating fashion (the lower-middle ellipse) in Fig. 7. Suppose 
the top and bottom segments are to be reconnected while the left and right segment 
remain. Then one cannot connect the upper-left W oo point with the lower-left T 
oo point, as it would cause the ellipse to be broken into two topological disks, a 
structurally unstable case. This means that it is not possible to have a reconnection 
bifurcation that takes this scenario to itself. Instead, the only way to reconnect is to 
connect the upper-left W oo point with the lower-right W oo point and connect the 
upper-right T oo point with the lower-left T oo point, leading to the scenario of two 
adjacent W and two adjacent T oo points on the ellipse (the upper-middle ellipse). 

In fact, the above argument is true in general. That is, when reconnecting, the 
upper-left oo point must be connected to the lower-right oo point, and the upper- 
right oo point must be connected to the lower-left oo point. This analysis shows 
that there are five degenerate curve reconnection bifurcations that takes one scenario 
to itself, and there are two more that occur between two different scenarios (the 
upper-middle ellipse and the lower-middle ellipse). 

Figure 8 shows the reconnection in the scenario of four WW curves, in which 
two linear degenerate curves (green) are reconnected, resulting in two linear degen- 
erate curves. The two planar degenerate curves (yellows) are not reconnected. This 
bifurcation does not increase or decrease the number of transition points. 

Figure 9 presents another degenerate curve reconnection bifurcation in which two 
planar WT degenerate curves are reconnected into one planar WW degenerate curve 
and one planar TT degenerate curve. As a result of this bifurcation, two transition 
points are removed, which leads to a decrease in the total number of transition points 
in the field. Notice the order of colored segments (middle column of Fig. 9) is changed 
due to the reconnection. 
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Fig. 7 This diagram shows all the possible degenerate curve reconnection bifurcations in a 3D 
linear tensor field 


Ker 


Fig. 8 This figure shows a degenerate curve reconnection bifurcation from the case of four WW 
curves (left) to four WW curves (right). The number of transition points does not change after the 
bifurcation 


Kes 


Fig. 9 Another degenerate curve reconnection bifurcation takes the case of one WW curve, 2WT 
curves, and one TT curve (left) to four WT curves (right). In this case, the number of degenerate 
curve bifurcation is decreased by two 
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4.3 Transition Point Pair Cancellation and Generation 


The total number of transition points in a 3D linear tensor field is an even number 
between zero and eight. That means that any fundamental bifurcation that changes 
the number of total transition points must either increase it by two or decrease it by 
two. The former is the degenerate curve pair generation bifurcation, and the latter 
degenerate curve pair cancellation bifurcation. They are inverse bifurcations and 
thus discussed together in this section. 

Two examples of transition point pair cancellation bifurcations are shown in 
Figs. 11 and 12, for the cases of two degenerate curves and four degenerate curves, 
respectively. The transition point pair cancellation and generation bifurcations do 
not change the scenario. Consequently, for each of the nine scenarios, there is a 
bifurcation that takes this scenario to itself, as shown in Fig. 10. There is a total 
of nine transition point pair cancellation bifurcations and nine transition point pair 
generation bifurcations. 


w T 


C? 
C> 


T w 


Fig. 10 This diagram shows all the possible transition pair cancellation and generation bifurcations 
in a 3D linear tensor field for one degenerate curve 


$> 


Fig. 11 A transition point pair cancellation bifurcation that removes two transition points and 
one blue segment from one of the two degenerate curves (left), resulting in the same number of 
degenerate curves with two fewer transitions points (right) 
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Fig. 12 A transition point pair cancellation operation (from to right) occurs in the case of four 
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Fig. 13 This diagram shows all the possible transition point relocation bifurcations in a 3D linear 
tensor field. Note that these bifurcations have the effect of switching one W symbol to one T 
symbol, or vice versa. Consequently, there are only eight possible such bifurcations, all observed 
in our experiment 


4.4 Transition Point Relocation 


Transition points, along with oo points, divide the degenerate curves into segments 
of pure L/P and W/T types. While the total number of transition points in a 3D linear 
tensor field is eight, its distribution is not uniform among the degenerate curves. 

Transition point relocation refers to moving one transition point from one degen- 
erate curve to another degenerate curve of the opposite L/P type. From the viewpoint 
of the parameterization, this is the swap of the positions of the transition point with 
one adjacent infinity point on the ellipse in the counterclockwise order. This results 
in one fewer transition point on the original degenerate curve and one more transition 
point on the new degenerate curve but keeps the total number of transition points 
constant. Moreover, the W/T type of the segment also changes. Consequently, the 
segment between the transition point and infinity point will change its L/P type but 
maintains its W/T type, thus will change colors between red and green, or between 
yellow and blue. Figures 14 and 15 provide two examples of this type of bifurcations 
that involve respectively two degenerate curves and four degenerate curves. 
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L387 


Fig. 14 This figure shows a transition point relocation bifurcation from the case of two WW curves 
(left) to two WT curves (right) 


hen 


Fig. 15 Another transition point relation bifurcation takes the case of four TT curves (left) to two 
TT and two WT curves (right) 


Figure 13 shows the possible transition point relation bifurcations. Note that these 
bifurcations result in a switch of a W infinity point to a T type of infinity point, or 
vice versa. Note that each edge in this diagram corresponds to two transition point 
relocation bifurcations. As there are eight edges in the diagram, there is a total of 16 
bifurcations, all of which have been observed during experiment. 


5 Conclusion 


Degenerate curves and transition points play an important role in describing the 
behavior of a 3D tensor field. In this book chapter, we study possible bifurcations 
for any 3D linear tensor field. 

This is done based on the realization that there is a total of nine scenarios and four 
types bifurcations. Theoretically, we have identified ten degenerate curve removal 
bifurcations, ten degenerate curve generation bifurcations, seven degenerate curve 
reconnection bifurcation, nine transition point pair cancellation bifurcations, nine 
transition point pair generation bifurcations, and 16 transition point relation bifurca- 
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tions. This leads to a total of 61 topological bifurcations for 3D linear tensor fields, 
all of which have been observed. 

There are a number of future research directions. 

First, the number of transition points can further divide each scenario into sub- 
scenarios. For example, for the scenario of two W W curves, there can be zero, two, 
four, six, or eight transition points. Furthermore, transition points are not evenly dis- 
tributed on degenerate curves. This leads to multiple cases per scenario. Enumerating 
of all bifurcations from any sub-scenario to any other sub-scenario can lead to deeper 
insight about the topology of 3D symmetric tensor fields. 

Second, the bifurcations among 3D tensor fields can be used to generate a multi- 
scale framework for the topological analysis of these objects, such as degenerate 
curve clustering and removal. Such a framework has the potential of enabling users to 
inspect the topology of their tensor field data at various scales. We plan to investigate 
this possibility in our future research. 

Third, simulation data sets usually involve piecewise linear tensor fields, which 
can have potentially more types of bifurcations than linear tensor fields that we 
investigate in this paper. This is a future direction that can lead to more practical 
applications for tensor field topology. 

Fourth, neutral surfaces are the other important constituent of 3D tensor field 
topology. Enumerating the scenarios of neutral surfaces and bifurcations of these 
different scenarios can provide insight into the structure of a tensor field. We will 
explore this direction. 
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Continuous Histograms for Anisotropy A) 
of 2D Symmetric Piece-Wise Linear ches 
Tensor Fields 


Talha Bin Masood and Ingrid Hotz 


Abstract In this chapter we present an accurate derivation of the distribution of 
scalar invariants with quadratic behavior represented as continuous histograms. The 
anisotropy field, computed from a two-dimensional piece-wise linear tensor field, is 
used as an example and is discussed in all details. Histograms visualizing an approx- 
imation of the distribution of scalar values play an important role in visualization. 
They are used as an interface for the design of transfer-functions for volume render- 
ing or feature selection in interactive interfaces. While there are standard algorithms 
to compute continuous histograms for piece-wise linear scalar fields, they are not 
directly applicable to tensor invariants with non-linear, often even non-convex behav- 
ior in cells when applying linear tensor interpolation. Our derivation is based on a 
sub-division of the mesh in triangles that exhibit a monotonic behavior. We compare 
the results to a naive approach based on linear interpolation on the original mesh or 
the subdivision. 


1 Introduction 


Iso-contours or iso-surfaces of scalar fields, defined as the pre-image of an iso-value, 
play a central role in visualization. There is a large body of work centered around 
iso-contour computation and analysis with many applications. A prominent exam- 
ple is the contour tree, which keeps track of topological changes of iso-contours 
when changing the iso-value [5]. It provides a structural overview of the data, see 
Sect. 2.3 for more details. Complementary information is provided by the continuous 
histogram, which is an extension of the discrete histogram to encode the distribution 
of the scalar values of continuous functions, see Sect. 2.1. It can serve as a valuable 
quantitative signature [2] of a data set. Both structures are frequently used for inter- 
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Fig. 1 An example demonstrating the behavior of the anisotropy field in a simple tensor field 
given on a triangulated domain. The ellipses at the vertices represent the tensors defining the field. 
The total rotation of the eigenvector when moving once around the domain is 277. The white lines 
represent iso-contours. a The anisotropy is linearly interpolated within the original mesh. It only 
has one minimum and no zeros or isotropic points. b The consistent anisotropy assuming linear 
interpolation of tensor components has three minima and two isotropic points. The respective join 
trees for the anisotropy fields are shown next to the meshes. All zero-leaves, marked in red, in the 
tree represent isotropic points of the tensor field. c The histograms for anisotropy using the two 
approaches in comparison 


action and filtering, for example, to determine interesting iso-values or for transfer 
function design [19]. Histograms have also been applied for tensor field visualization 
and exploration to display the distribution of scalar invariants derived from tensor 
fields [13]. This distribution can also be understood as statistics of iso-contours. Note 
that we use the term iso-contour to refer to the complete level set of a 2D scalar field 
for a given iso-value, not one of its connected component which is the case in some 
of the related contour tree literature. 

There are many standard algorithms to compute and analyze iso-contours, which 
are mostly based on piece-wise linear or multi-linear behavior. This assumption is, 
however, often violated for tensor invariants when using component-wise tensor 
interpolation, for example, various anisotropy definitions or the determinant of the 
tensor. An illustrative example of the behavior of the contours of the anisotropy 
in piece-wise linear tensor field in comparison to a linear approximation of the 
anisotropy is shown in Fig. 1. It is important to note that, there is no continuous 
tensor field interpolating the tensors at the vertices exhibiting the piece-wise linearly 
interpolated anisotropy in Fig. la when assuming that the tensor field is fixed on 
the boundary of the domain. The number of rotations of the eigenvector field when 
moving once around the domain (Poincaré index) is a topological invariant that 
determines the minimum number of isotropic points in the field. In this example, 
there is a total rotation of 27 of the eigenvectors, which results in two zeros in 
the anisotropy field. These zeros in the anisotropy field are not present in the linear 
anisotropy field corrupting its histogram, Fig. 1c. Therefore, it is important to consider 
an anisotropy field which is consistent with the chosen tensor field interpolation 
when analyzing its behavior. Here we have chosen a piece-wise linear interpolation 
of tensor components as an example. In the following we refer to the anisotropy 
field resulting from this interpolation as the consistent anisotropy. In this chapter, we 
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present the derivation of an exact closed-form expression for continuous histograms 
for quadratic tensor invariants that is consistent with a piece-wise linear interpolation 
of a tensor field given over a two-dimensional triangulated domain. 

Here, we are especially interested in the analysis of the anisotropy, an important 
characteristic in many applications, defined as the difference between major and 
minor eigenvalue. While there are many other definitions of anisotropy, this version 
has a distinguished meaning when analyzing the stability of the critical points and 
isotropic points, when perturbing the tensor field using the Frobenius norm to mea- 
sure the deviation of the tensor field [18]. Further, it is frequently used in mechanical 
engineering applications to analyze the failure of materials. The join tree, a subset 
of the contour tree, of this anisotropy can be used for a topology-preserving simpli- 
fication of 2d tensor fields [11]. 


Our main contributions are: 


e Framework for the analysis of scalar invariants that is consistent with a piece-wise 
linear interpolation of the tensor components. This includes topologically correct 
iso-contours, the contour tree, and continuous histograms. 

e A generic closed-form formulation of histograms for continuous data based on a 
generalization of the cumulative histogram. This approach neither requires explicit 
computations of the length of iso-contours nor the computation of the gradient. 

e Explicit solution for the histogram for piece-wise quadratic functions with positive 
Hessian over a two-dimensional domain. The anisotropy over a piece-wise linear 
component interpolation provides a relevant example. 

e Comparison to naive approaches using a linear interpolation of the anisotropy and 
a discussion of the results. 


The relevant context and related work are summarized in Sect. 2, then an overview 
of the problem and its solution is given in Sect. 3. Fundamental definitions and nota- 
tions are introduced in Sect. 4. A general expression for the anisotropy is developed 
in Sect. 5 and used in Sect. 6 to subdivide the domain in monotonous triangles. 
Section 7 derives the accurate histogram. After some notes on the implementation 
in Sect. 7.1 the results are discussed in Sect. 8. 


2 Context and Related Work 


In this section, we first summarize the development of the concept of histograms 
from a signature of discrete data to continuous fields. Then we summarize some 
aspects of tensor interpolation and invariants that motivated our work. 
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2.1 Continuous Histograms 


Originally, histograms have been developed for discrete data. They go back to Pearson 
and refer to a graphical representation displaying the frequency of the data items as 
bars over the data range [10]. As such, a histogram provides a summary of the data 
and they are also used to compare and characterize data sets. Formally, the histogram 
for a discrete set of data values F = { f;|i € 1,--- , m}isaone-dimensional bar chart 
displaying the frequency of each distinct element f € F as a bar with height 


h(f) =o 6(f - fi) 


where ô is the delta function which is equal to one if f = f; and zero otherwise. A 
related concept applicable to ordered data is the cumulative histogram which counts 
the cumulative number of data values smaller than a given value f € F. This results 
in values 
c= > nf. 
fisf 


Today, the term is often not only used for the graphical representation but also for 
the concept capturing the distribution of function values by binning the function 
range and counting the frequency of data samples within these bins. The resulting 
histograms, however, are strongly dependent on the binning size and the sampling 
strategy of the function in the domain. The continuous nature of the function between 
the sample points is not represented. For these reasons several concepts to extend 
histograms to continuous functions have been proposed resulting in a distribution of 
the function values. Bajaj et al. [2] have introduced the contour spectrum as a data- 
signature to find interesting iso-values for volume rendering. The contour spectrum 
assigns the geometric measures of the contour length or surface area to each scalar 
value. In addition, they consider areas and volumes of regions below or above a 
given iso-value. Bajaj et al. also propose a method to compute these values exactly 
under the assumption of a piece-wise linear interpolation and a piece-wise constant 
gradient. Later Carr et al. [4] investigated the relation between histograms and the 
contour spectrum based on a nearest-neighbor interpolation. Scheidegger et al. [16] 
completed this work and introduced a natural generalization of histograms to the 
continuous setting as the distribution given by the contour spectrum weighted by the 
local isosurface density expressed as the inverse gradient magnitude. Given a scalar 
field f and scalar value ¢ the distribution is given as 


o=] wfayrtas 
fo) 


For the derivation of the distributions, they refer to Coarea formula that provides a 
relationship between the sum of area integrals and a global volume integral as used 
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by Mullen et al. [14]. For piece-wise linear functions, this yields a more or less good 
approximation of the real distribution. 

Similar concepts have also been considered for multivariate fields, for example, 
generalizations of scatterplotts [1] and parallel coordinates [9] to continuous data. 
In both cases, the mathematical model is based on mass conservation of a map- 
ping from the function range to the scatterplot space and the parallel coordinates 
space respectively. The continuous histogram is a special case considering a one- 
dimensional range. While the mathematical model is generic, the proposed compu- 
tational algorithms are limited to specific interpolation models in the spatial domain. 
The algorithm by Bachthaler et al. [1] assumes a linear interpolation in tetrahedral 
cells. 

In our work, we focus on invariants with a quadratic behavior and propose an 
exact solution pursuing a slightly different approach than most of the previous meth- 
ods. Instead of directly generalizing the concept of histograms, we start with the 
generalization of the cumulative histogram. This approach requires neither surface 
area/length of iso-contours nor gradient approximations and thus makes it possible to 
compute an accurate distribution for more complex interpolation schemes, demon- 
strated here for quadratic interpolations. The continuous histogram or scalar value 
distribution then follows as a derivative of the cumulative histogram. 


2.2 Notes on Tensor Field Interpolation 


In this section, we briefly summarize some notes related to tensor field interpolation 
without going much into detail. This has been a frequently discussed topic in many 
applications and it is agreed that this is a challenging topic. 

While many theories and visualization models assume data given as fields on 
continuous domains, the data-reality are data sets sampled at discrete locations. 
Depending on the origin of the data this can be voxel-based data from imaging or 
meshes for simulation data. In any case, itis necessary to approximate and reconstruct 
the field from these samples. Thereby the most commonly used methods for tensor 
fields are based on a component-wise interpolation. Similar to other applications 
and other data types these are linear, bi-, or tri-linear interpolation depending on the 
grid type. Concerning such interpolations, a frequently discussed topic is the non- 
linear dependency of most tensor invariants on the tensor components [12]. There 
have been many methods proposed which explicitly try to preserve certain tensor 
characteristics [3, 17]. Recently there appeared a survey on interpolation methods 
for positive definite tensors [8]. 

We will, however, stay with the most simple interpolation methods, linear 
component-wise interpolation within tetrahedra which is also often the basis for 
finite element simulations. The non-linear dependence of the tensor invariants can 
lead to a non-convex behavior of the invariants inside the cells. An example we are 
especially interested in is the behavior of the anisotropy, defined as the difference 
between the maximum and minimum eigenvalue, which is typically decreased inside 
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the cells. Sometimes this is referred to as “swelling-artifact”’, but we rather argue that 
this is a fundamental property of tensor behavior. It reflects the fact that the anisotropy 
is linked to the directional behavior of the tensor. This is expressed by the fact that 
the anisotropy is always zero at degenerate points. These are points where the eigen- 
values are the same and there is no uniquely determined eigenvector direction. The 
number of degenerate points is a topological invariant of the field when fixing the 
boundary and thus determines the minimum number of zeros in the anisotropy field, 
see Fig. 1 for an example. Therefore the analysis of the anisotropy field must be 
handled consistently with the chosen interpolation method for the tensor field. This 
concerns the computation of iso-contours, the contour tree and also histograms. 

For our consideration in this work, we assume that we have a continuous tensor 
field given on a two-dimensional triangulated domain. We further assume a piece- 
wise linear, component-wise linear interpolation of the tensor field within these 
triangles. The anisotropy then has a quadratic behavior in the triangle. One can 
observe similar behavior for other invariants, for example, the determinant too. All 
the above-proposed methods for histogram computations fail when the interpolation 
of the data is not convex. Similarly, typical iso-contour or contour tree computation 
is also based on convex interpolation. 


2.3 Contour Trees, a Topological Summary of Scalar 
Functions 


The contour tree keeps track of the changes of sub- and super-level sets of a function, 
bounded by iso-contours, and thus provides a valuable summary of the function, 
compare Fig. 2. It can be assembled from the join tree tracing the changes in iso- 
contours when the function value is increased from —oo to oo and the split tree one 
where the function value is decreased from oo to —oo. In more detail, the join tree 
tracks the creation and merging of sub-level sets, which are recorded in a tree. At 
the local minima of the function, new branches are created. As the function value 
increases, branches are extended and merged at saddle points where two sub-level 


M 


ve Join Tree Split Tree Contour Tree 


Fig. 2 For the scalar field on the left, the join tree, split tree and the contour tree are shown 
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sets merge. The global maximum of the function becomes the root of the tree. The 
split tree is constructed similarly while traversing the field from oo to —oo. The 
most commonly used algorithm for contour tree computation assumes a piecewise 
linear interpolation of the scalar field where all critical points are located at the 
cell vertices [5]. In the case of the anisotropy, we are especially interested in the 
join tree, a sub-tree of the contour tree, since many of the minima correspond to the 
degenerate points in the tensor field. The join tree can be used to quantify the stability 
of these points [18]. For consistent results, assuming a specific interpolation, a join 
tree computation based on the quadratic behavior or the anisotropy is essential. 


3 Problem Statement and Solution Overview 


Given a tensor field sampled over a 2D triangular mesh, our goal is to compute the 
accurate contour-tree and histogram of the tensor anisotropy consistent with linear 
interpolation of tensor components. Anisotropy is a quadratic function under this 
interpolation. 

Figure 3 gives an overview of our solution approach. For each triangle in the input 
mesh, we first determine the coefficients of the quadratic function for the anisotropy 
based on tensor values at the triangle vertices. We show that anisotropy is a special 
quadratic function which either has a single minimum equal to zero and elliptical 
iso-contours, or in a degenerate case, minima along a line and the iso-contours are 


Input Triangle Monotonous Triangles Histograms 


Cumulative Histogram 


Normalization | L 


Histogram 


la 


Fig. 3 Solution overview: First, the coefficients of the quadratic function are determined. Then a 
transformation is applied such that the minimum is at the origin and the iso-contours are circular, 
followed by a sub-division into monotonous triangles. For each monotonous triangle, the cumulative 
anisotropy histogram is computed using sub-level set areas, which are added to get the cumulative 
histogram for the original triangle. Finally, the derivative of the cumulative histogram yields the 
anisotropy histogram 


Quadratic anisotropy 


function Subdivision 
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pairs of parallel lines, refer to Sect. 5. The first step is the subdivision of the triangle 
into monotonous sub-triangles, which is the basis for the iso-contour extraction, the 
contour tree computation and the derivation of the histogram. We obtain the histogram 
as the derivative of the cumulative histogram, which is computed first from the area 
of the sub-level set for the isovalues. To compute the explicit area of the sub-level 
sets we apply a linear transformation to the coordinate system such that elliptical 
iso-contours become circular centered at the origin, refer to Sect. 5.1. 


4 Background and Notations 


In this section, we provide the notations and a brief mathematical background 
required for the proposed method discussed in sections later. First, in Sect. 4.1 we set 
up the notations for tensors and the invariant of interest to us that is tensor anisotropy. 
Then in Sect. 4.2, we describe the barycentric interpolation within a triangle, since 
this approach is used for linear interpolation of tensors within a triangle. Lastly, in 
Sect. 4.3, we discuss the general bi-variate quadratic function, its critical point, and 
the shape of its iso-contours. 


4.1 Second Order Symmetric Tensors and Anisotropy 


: e ; 
For a second order symmetric tensor, T = ( f 4 ) the two eigenvalues are: 


2 

perp) D ep 

Depending on the application different measures for anisotropy are used. For positive 
definite tensors, relative measures like the fractional anisotropy are common. In 
mechanical engineering a typical measure is the difference between the maximum and 
the minimum eigenvalues a(T) = A — u = y (e + g)? — 4(eg — f°). It has been 
shown that this value is also related to the stability of degenerate points in tensor 
field topology [11] and is the measure we are mostly interested in. In the following, 
we will however consider the squared value of æ (T), which has the same topological 
characteristics but simplifies the computations a lot. 


v(T) =° (T) = (A- py’ 
(e—g) +4f? (1) 
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4.2 Barycentric Coordinates and Piece- Wise Linear 
Interpolation 


For all our considerations in this paper we use a component-wise linear interpolation 
of the tensor field. We use barycentric coordinates for interpolation in the triangle. 
Since we require an explicit representation of the the tensor field for the derivation 
of the exact histogram we briefly review the main definitions and formulas in this 
section. 

Refer to Fig. 4, for a triangle with vertices pı = (x1, y1), p2 = (x2, y2) and p3 = 
(x3, y3), the barycentric coordinates (£1, 62, 63) of an arbitrary point p = (x, y) 
within a triangle are given by 


O2 = ys) (% = x3) + (Ga = x2)(y — y3) 
(y2 — y3)(x1 — x3) + (x3 — x2) (1 — y3) 
Q3 — yi) — x3) + (1 — x3) — Ys) 
O2 — y3)(x1 — x3) + (x3 — x2) (1 — y3) 

o Qiy) = X2) + 2 — x1) — Ya) 
(yo — ya — 23) + (3 — 22) (1 — 3) 


B= 


po = 


We assume that some scalar function s is sampled at the vertices of the triangle with 
scalar values s1, 52 and s3 at vertices pı, p2 and p3. Then, the scalar value s at an 
arbitrary point p = (x, y) within the triangle can be determined as 


s(x, y) = Bis, + Bos2 + B353. 


This function s(x, y) is linear in x and y, and can be alternatively written as 


S(x, Y) = SxX + Syy + Sc, 


Fig. 4 Barycentric coordinates and interpolation. The point p within the triangle Ap, p2 p3 
can be represented using barycentric coordinates (61, 62, 3), where 6, = Area(App2p3)/ 
Area(A pı p2p3), Bz = Area(Appıp3)/Area(A pı p2p3) and 3 = Area(Appı p2)/Area 
(Apıp2p3). Let s be some scalar quantity sampled on the vertices pı, p2 and p3 as s1, 52 
and s3 respectively. Then the value of s at any point p inside the triangle is given by 
s(p) = isı + p282 + B353 
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L (2 — y3)51 + O3 — W182 + Q1 — ya)53 
t (y2 = y3) — x3) + (3 — x2) 1 — ys) 
_ (%3 — x2)s1 + (x1 — 23) 82 + (x2 — 21) 53 
(2 = y3) — x3) + (83 — 2) — ys) 
i (X23 — x3 Y2)51 + (X31 — X1 y3)82 + 12 — %2Y1)83 
° (y2 — y3) (x1 — x3) + (x3 — x2)(y1 — Ys) 


4.3 Bivariate Quadratic Functions and Their Critical Points 


As the anisotropy, as defined in Eq. (1), is a quadratic function we summarize some 
general facts about quadratic functions and define the notation that we will use later 
in the paper. In the following we assume that the quadratic function is not identical 
to zero. 

A general quadratic function in two variables can be written as s(x, y) = Ax? + 
Bxy + Cy? + Dx + Ey + F or in matrix form as 


saem (Ver. E) +r D 
; _{ A B/2 _(D ioe ‘ : : 
with M = ( B/2 C ) and T = ( a The critical point of the scalar function s is 


a point p = (x, y) where the gradient Vs(x, y) = 0 is zero. The partial derivatives 
of s(x, y) with respect to the two variables is: 


ðs/ðx =2Ax + By + D (3) 
ds/Oy=2Cy+ Bx + E (4) 


The location of the critical point pe = (xe, Yc) of s can be obtained after solving 
the linear equations ðs /ðx = 0 and ðs/ðy = 0 using Eqs. (3) and (4). The resulting 
coordinates are 


_ -2CD+ BE _ -2AE+BD 


c= Pe 5 
*e = “AAC — B? Ye = ZAC — B? () 


The function s and its critical point can be classified based on the sign of the 
determinant of the Hessian, H 


H =4AC — B?. (6) 


If H > 0, then the critical point is either a maximum or minimum and the iso- 
contours are ellipses, compare Fig. 5a, b. The type of the critical point depends on 
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(a) (b) (c) 
(d) (e) (f) 


Fig. 5 The possible cases for a general quadratic function. a The critical point is a minimum and 
the iso-contours are elliptical. b The critical point is a maximum and the iso-contours are ellipses. 
c The critical point is a saddle and the iso-contours are hyperbolas. d The critical point does not 
exist and the function has a minimum value along a line, while the iso-contours are a pair of parallel 
lines. e The other case when the iso-contours are a pair of parallel lines, again the critical point does 
not exist but the function has a maximum value along a line. f The case when the critical point does 
not exist and the iso-contours are parabolic. We show that only the cases (a) and (d) are possible 
for the quadratic function for tensor anisotropy, the other four cases shown here are not possible 


the sign of A and C. If A, C > 0, then the critical point is a minimum, otherwise it 
is a maximum. In the other case when H < 0, then the critical point is a saddle and 
the iso-contours are hyperbolas. 

For the case when H = 0, there are no isolated critical points and the iso-contours 
are either parabolas, parallel or coincident lines. Note that it is possible that there are 
degenerate lines. The type of the iso-contour when H = 0 depends on the invariant 
I defined as 

I = BDE — AE’ —CD”. (7) 


If J Æ 0, the iso-contours are parabolic, compare Fig. 5f, otherwise they are pair 
of parallel lines, compare Fig. 5d, e. 
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5 Anisotropy for 2D Piece-Wise Linear Tensor Fields 


In this section, we have a closer look at the anisotropy function for a linear tensor 
field. The main observation is in the non-degenerate case, that the anisotropy always 
has exactly one minimum with a value of zero. This corresponds to the fact that the 
tensor field always has exactly one critical point. All iso-contours are ellipses. 
Given a triangle with vertices pı = (x1, y1), p2 = (x2, y2) and p3 = (x3, y3). Let 
the tensors at these vertices be T, T and T3, respectively. Let the components of the 


tensors be 
e fi e fr e3 fs 
Ti = , bh= ; h= ; 8 
i G A i £) f (i ) (8) 
The barycentric coordinates can be used for linear interpolation of the tensor field 
within the triangle. Tensor T at an arbitrary point p = (x, y) within the triangle can 


be found as: 
e(x, y) FQ, a) 9) 


Mp) = PG, y) = Gs y) g(x, y) 


As described earlier in Sect. 4.2, the tensor components e, f and g linearly inter- 
polated within the triangle as can be written as 


e(x, y) = exx + eyy + ec, (10) 
f@,y)= fax + fy + fe, (11) 
(X,Y) = BxX + ByV + ke. (12) 


The function, we are interested in is the anisotropy v, the explicit expression for 
which can be obtained by substituting the linear expressions for tensor components 
in Eq. (1) yielding 


v(T) =((ex — gx)” +4 f2)x7 + 2((ex — Bx)(ey — By) + 4 fx fy) xy 
+ (Cey — 8y)? +46 )y? + 2((ex = 8x) (Cc — ge) + 4 fx fe) x 
+ 2((ey — gy)(ec — 8e) +4 fy fe)y + (lec = Be) +4f2). 03) 


Comparing Eq. (13) with the general quadratic Eq. (2), we obtain the coefficients of 
the quadratic function in dependence of the tensor components: 


= ((ex — gx)” +4 f2) = 0, (14) 
= 2((ex — gx)(ey — 8y) + 4 fx Fy). (15) 
(ey — gy) +4f;) = 0, (16) 
(ex — Bx) (Cc — 8c) +4 fx fe): (17) 
(ey — gy)(@e — 8) + 4 fy fe)» (18) 


A= ( 
B=2 
C = ( 
D=2 
E=2 
F = (lee — gc)’ +4 f2). (19) 
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Substituting the above equations in Eq. (6), we derive the determinant of Hessian for 
the anisotropy to be (for derivation refer to Appendix A): 


H =16( fx (ey = gy) = Sy (ex = go) > 0. (20) 


From Eq. (20), we can conclude that the iso-contours of anisotropy function are 
never hyperbolic. In most cases, when H is strictly greater than 0, the contours are 
elliptical. Moreover, in that scenario, from Eqs. (14) and (16), we can deduce that 
the type of the critical point is always a minimum. 

Appendix A contains the detailed analysis of v. We show that the bi-variate 
quadratic function v is a special function that has elliptical iso-contours or in some 
cases a pair of parallel lines, ruling out the possibility of hyperbolic or parabolic 
iso-contours. 


5.1 Field Normalization Using Coordinate Transformations 


In the following, we derive a coordinate transformation such that the bivariate 
quadratic scalar function determined for a triangle has a standard format. This allows 
for the application of a unified strategy for further analysis. 

Equation (13) gives the expression of anisotropy in general bivariate quadratic 
form. We have also established that iso-contours of this function will are elliptical. 
Therefore we first transform the coordinates such that elliptical iso-contours are 
centered at the origin and aligned to the axes. This can be achieved by applying a 
translation and rotation, both rigid-body transformations preserving areas. Then, a 
transformation is applied such that iso-contours of the bivariate quadratic function 
become circles rather than ellipses. This can be achieved by applying a non-uniform 
scaling, a linear transformation which distorts the area by a constant factor given 
by the determinant of the transformation matrix. The approach described above is 
illustrated in Fig. 6. 

Refer to Eqs. (13)-(19) for the detailed expression of anisotropy and refer to 
Eq. (2) for the general quadratic equation in matrix form. After translation such that 
the minimum pe = (Xc, Yc) falls into the origin, the expression for anisotropy can be 
written as 


V(X, Y) = Xr, Y) M. (*) +F, (21) 


with the translated coordinates x; =x — Xe, yy=y—ye and Fy =F + 
BPE A. Using Eqs. (14)-(19), it can be shown that F, = 0 for v. This implies 
that minimum value of v is zero at the critical point and this point is a degenerate 


point in the tensor field. Equation (21) becomes 


væn y) = Op y) M- (z) (22) 
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(a) (b) (c) (d) 


Fig. 6 Field normalization using coordinate transformations. a The original quadratic function, 
f(x, y) = x? + xy + y? + 0.4x + 0.5y + 0.07. b The function after applying translation such that 
the minimum is at the origin, f (xr, yr) = x? + x1); + y?. ¢ After applying rotation such that ellip- 
tical iso-contours are axes aligned, f(x», y-) = 0.5x2 + 1.5y?. d After scaling such that elliptical 
iso-contours are transformed into circular iso-contours, f (xs, ys) = x2 + y? 


Now, we apply a rotation O = (EV,|E V3) to align the elliptic iso-contours with 
the axis using the eigenvectors EV, and EV of the matrix M 


Xr Xt 
=0. : (23) 
J C) 
which results in the diagonal representation of the anisotropy 


V(X, Yr) = 1X? + Agy? (24) 


where à; and A> are the eigenvalues of M. 
Lastly, we apply non-uniform scaling to obtain circular iso-contours: 


VEs, Y) =x? +y? where xs = yax Ys = VAY (25) 


The area distortion because of this scaling is given by the factor /A)A2. 


6 Subdivision in Monotonous Sub-triangles 


We consider a triangle with vertices pı = (x1, y1), p2 = (%2, y2) and p3 = (x3, y3) 
and the tensors at these vertices are T;, T> and T3. The tensors are linearly interpolated 
within the triangle and the anisotropy is a quadratic function. For the extraction of 
iso-contours, the computation of the contour tree as well as for the histogram we 
require piece-wise monotonous behavior inside the triangles, which is given in this 
chapter. Similar, subdivisions have also been proposed by Dillard et al. [6] and by 
Nucha et al. [15]. There are five different cases depending on the location of the 
global minima and the local minima at the edges. Especially the cases when the 
minimum p, lies within the triangle or outside the triangle need a different treatment 
for the computation of the anisotropy histogram. 
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(i) j (k) 


Fig. 7 All the possible cases and the corresponding sub-divisions into monotonous triangles. a, b 
The case when the minimum is inside the triangle. We can subdivide the triangle into six monotonous 
triangles. c, d The case when the minimum is outside the triangle, but the function restricted to the 
triangle edges has a minimum on all the edges. Four monotonous triangles can be generated. e, f 
The case when two of the edges have minima, three monotonous triangles are generated. g, h The 
case when only one triangle edge has a minimum, two monotonous triangles are created. i, j The 
case when none of the edges has a minimum, no triangle subdivision is required. k, 1 Lastly, the 
degenerate case when the function does not have a point minimum but has a degenerate minimum 
along a line. In this case, the iso-contours are a pair of parallel lines 


Case A. The minimum p, is within the triangle. In this case, the input triangle can 
be partitioned into six sub-triangles such that v behaves monotonously 
within the triangle. See Fig. 7b. 

Case B. The minimum pe is outside the triangle. Here, we have four possibilities: 


1. No triangle edge has an edge minimum. See Fig. 7i. 

2. One triangle edge has an edge minimum. See Fig. 7g. 
3. Two triangle edges have edge minima. See Fig. 7e. 

4. All three triangle edges have edge minima. See Fig. 7c. 


In all these cases, the triangle can be sub-divided into an appropriate number of 
monotonous triangles as described in Lines 14-30 of Algorithm 1. Although these 
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sub-divided triangles are monotonous and look similar to Case A, an important 
difference is that in general none of the vertices lies at the origin. 


7 Computation of the Histogram for v 


In this section, we derive the exact continuous histogram of the anisotropy for linearly 
interpolated tensor fields. We use the term histogram here not strictly as it has been 
originally introduced but in the sense of the distribution of function values. It can 
also be considered as the weighted contour spectrum using the terminology of Bajaj 
et al. [2] as introduced by Scheidegger et al. [16]. In contrast to previous methods, 
we approach the problem by first computing the cumulative histogram CH and 
then derive the histogram from it by computing its derivative. This makes the exact 
computation of the histogram much more feasible. 


CH (v) = Area(v < vo) = È Areaj(v < vo) 
Ai 


where Area;(v < vo) is the area of the sublevel set in triangle A;. Specifically, we 
consider here the anisotropy with its quadratic behavior, however the method also 
directly applies for linear fields. 

In our derivation of Area;(v < v9) we consider the following setting. Given is a 
triangle AABC with vertices A = pı = (x1, y1), B = p = (%2, y2) and C = p3 = 
(x3, y3) and respective tensors Ti, D) and 73, which are linearly interpolated within 
the triangle. Further we ordered the vertices such that the anisotropy values are 
vı < v < v3, using simulation of simplicity [7] to resolve the ties if required. To 
obtain the contribution of the triangle to the cumulative histogram at the value v 
we compute the area of the sublevel set at v within the triangle. In the following, 
we assume that anisotropy is monotonous within the triangles resulting from the 
subdivision introduced in Sect. 6. We further assume that the transformations as 
described in Sect. 5.1 have been applied. This means that the anisotropy has the form 
as given in Eq. (25) with a global minimum p, = (0, 0) at the origin and the level sets 
that are circles. The area within the transformed triangle is distorted by a constant 
factor, which can be appropriately multiplied to get the exact areas for the original 
triangle. We consider two cases depending on whether the vertex A of the triangle 
vertex lies at the origin or not. 


Case 1: The global minimum p, lies at one triangle vertex 


Let’s assume that the global minimum p, lies in vertex A = p; of the triangle ABC. 
Then the shape of the sublevel set for v can have two different types depending on 
whether v is smaller or larger than v = a g- Ifv < v the sublevel set is a sector of 
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a circle with radius r = ./v and opening angle 04, as shown in Fig. 8. The area of 
the sublevel set for v € [v1, v2] is then given by 


Oav 
Area(v) = DE (26) 
where 04 is angle between the edges AB and AC. The rate of change of the area is 
given by 


ðArea(v) = Oa (27) 
ðv 2 
which is a constant not depending on the specific value of v. 

If the isovalue v is greater than v2 and less than v3, the iso-contour is composed 
of a circular sector and an additional more complex shape DB FE as illustrated in 
Fig. 8. This region is enclosed by two circular segments DB and EF and two line 
segments DE and BF and can be computed as 


Area(BDEF) = Area(BGHF) + Area(FHE) — Area(BGD) 
_ (= sin 04 +r sin 0)(r cos — rag =) 


2 
te 0r? rr? sin@ cos@ Garay rip sin 04 cos 04 
2 2 2 2 
Or? — Oar? in 64 cos@ — cos 04 sin@ 
_ or P88 rag ( E A COS — 4 sin | 


A=p, G DHE Cp 


Fig. 8 A triangle with monotonous function behaviour where the global minimum pe lies on one 
vertex A = pı. Due to the normalization of the triangle, pe is at the origin and the iso-contours are 
circular. If v < v2 the sublevel set is a subset of the light blue area. If v > v2 the sublevel area is 
composed of the complete light blue region and the area is highlighted in darker blue 
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With r? = v and ri g = v2 this results in 


6v —0 in(04 — 0 
Area(BDEF) = — + Joga), (28) 


In the expression above, the given value of v determines the values of 0. 


Case 2: The global minimum pc does not lie at any triangle 
vertex 


These triangles are the more generic case, we still assume that the anisotropy is 
monotonous inside the triangle and the vertices such that the anisotropy values are 
vı < v < v3. The triangles look similar to Case 1, but none of the vertices lie at the 
origin, compare Fig. 9. To compute the area of a sublevel set ABFE within these 
triangles, we considering the sublevel sets in two triangles where we can use the 
algorithm of Case 1. In Fig. 9 they are the triangles ADBC and ADHC. For the 
final area, we have in addition to consider the area of triangle AAH B which has to 
be added or subtracted depending on the exact position of A. 


Area(ABFE) = Area(DBFG) — Area(DHEG) + Area(AAHB) (29) 


Fig.9 A triangle AA BC with monotonous function behaviour where the global minimum pe does 
not lie on any vertex 
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7.1 Implementation 


We compute the cumulative histogram of anisotropy by computing the sublevel set 
area. The detailed algorithm is provided in Algorithm 1. For a given tensor mesh, we 
pre-process all the triangles to determine the coefficients of the quadratic function 
for anisotropy within the triangle, compare Sect. 5. Each triangle is appropriately 
subdivided into monotonous triangles, compare Sect. 6. Then for each anisotropy 
value in the histogram bin, we compute the sublevel set area by adding up the sub- 
level set area in all monotonous triangles. We also handle triangles with degenerate 
minimum appropriately. 


Parallelization. The algorithm described in Algorithm 1 has lots of options for 
parallelization. The loops at Lines 4 and 38 are embarrassingly parallel because 
each triangle in the mesh can be pre-processed in parallel. Similarly, each bin in 
the histogram can be computed in parallel. Further, with atomic add operations, 
the computation of the sublevel set area for anisotropy value v corresponding to a 
particular histogram bin can be parallelized over the set of monotonous triangles. 
These are the loops at Lines 41 and 44 in Algorithm 1. 


Numerical issues. Since there are a few floating-point operations and checks 
involved in the Algorithm 1, floating-point errors can be introduced which may prop- 
agate to the output resulting in a wrong histogram. We handle these errors by intro- 
ducing reasonable checks, for example making sure that the cumulative histogram 
thus obtained is always a monotonically increasing function and sums up to the total 
area of the domain for the largest value of anisotropy. However, a deeper study of 
the errors and more robust computation is left for future work. We are confident that 
using a multi-precision library for computation will remove the floating-point errors. 


Run times. We implemented the Algorithm | as a prototype in Java 1.8. This proof of 
concept implementation does not include extensive code and memory optimizations. 
The experiments reported in following Sect. 8 were performed on a workstation with 
12 core Intel Xeon processor and 32 GB of RAM. 

The running time depends on the size of the input mesh and the number of bins 
in the output histogram. For all the experiments we chose 500 as the number of bins 
in the histogram. The synthetic data used in Sect. 8.1 is a small data set with the 
mesh containing only 32 triangles. It took about 0.02 s to compute the continuous 
histogram for the meshes in this ensemble. The histogram computation for the two 
point load data reported in Sect. 8.2 took 0.2 s. This triangle mesh contained 1682 
triangles. The run time improved to 0.1 s when using multi-threading. Lastly, for the 
DTI data set containing 722 triangles (Sect. 8.3), it took 0.09 and 0.17 s to compute 
the histogram for brain and noisy regions respectively. The times improved by a 
factor of 2 for both cases when using multi-threading. 
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Algorithm 1: Compute cumulative histogram 


CeAmrArNnIa un kw nN = 


10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
31 
32 
33 
34 
35 
36 
37 
38 
39 
40 
41 
42 
43 
44 
45 
46 
47 
48 


Data: A tensor mesh M, and histogram resolution B 
Result: Cumulative histogram CH 


Initialize histogram CH to zeros; 
Initialize monotonous triangle list MT to be empty; 
Initialize degenerate triangle list DT to be empty; 
foreach triangle t in M do in parallel 


H = 4AC — B?; 
if H > 0 then 


else 
case 0 


case 7 


case 2 


case 3 


endsw 
endsw 
end 


else 

Add t to DT; 
end 

end 


CH [i] < 0; 


| CH [i] < CH{i] 


Add t to MT; 


for i < 1 to B do in parallel 
v < Range(MT)/i ; 


GetSubl 


Compute the quadratic function coefficients A, B, C, D, E, F; 


Compute the location of minimum pe = (xc, Ye); 

Compute the number and locations of edge minima; 
if pc is inside t then 
Generate six monotonous triangles from t and add to MT; 


switch Number of edge minima do 


Add an edge from edge minimum to opposite triangle vertex; 
This divides t in two monotonous triangles; 
Add the two monotonous triangles to MT; 


Add an edge between the two edge minima; 

Add an edge from lowest valued edge minimum to opposite vertex; 
These two edges divide t into three monotonous triangles; 

Add the monotonous triangles to MT; 


Find the lowest valued edge minimum pm; 

Add edges between pm and the two other edge minima; 

Add an edge from lowest valued edge minimum to opposite vertex; 
These three edges divides t in four monotonous triangles; 

Add the monotonous triangles to MT; 


foreach monotonous triangle t in MT do in parallel 


LevelSetArea (t, v); 


end 


foreach degenerate triangle t in 


DT do in parallel 


GetSubl 


LevelSetArea (t, v); 


| CH [i] < CH[é] 
end 


end 
return CH; 
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8 Results 


We apply the proposed approach of computing anisotropy histograms to three dif- 
ferent case studies. We show results for synthetic, simulation and experimental data. 
For all the case studies we compute the continuous histograms for three different 
interpolations of the anisotropy. 


e Interpolation [a]: Linear interpolation of the anisotropy within the original trian- 
gulation. 

e Interpolation [b]: Linear interpolation of the anisotropy within the sub-divided 
monotonous triangles, compare Sect. 6. 

e Interpolation [c]: Anisotropy based on the linear interpolation of the tensor com- 
ponents, resulting in a quadratic behavior or the anisotropy, compare Sect. 5. When 
using the term accurate we always refer to accuracy with respect to this interpo- 
lation. 


The resulting anisotropy fields are directly shown using a color map where dark blue 
refers to an anisotropy value of zero and red assigned to the maximum value. We also 
show a set of iso-contours in these images as white lines. In addition, we computed 
the join tree for all cases, which are always the same for interpolation [b] and [c]. 
Finally, we computed the continuous histogram where the results for one data set are 
plotted in one image, interpolation [a] displayed as a red curve, interpolation [b] as 
a green curve and interpolation [c] as a black curve. 


8.1 Synthetic Data 


The first example is a synthetic data set where tensors with user specified tensor 
components are placed at grid locations of a 5 x 5 grid. This grid is triangulated to 
provide a mesh with 32 triangles. This simple example is well suited to demonstrate 
the differences between the histogram of the consistent anisotropy field [b], and the 
methods utilizing a linear approximation of the anisotropy on the original mesh [a], 
and on the subdivided mesh with monotonous triangles [b], as shown in Fig. 10. It 
can immediately be seen that for interpolation [a] the topology of the iso-contours is 
not in accordance with the tensor field. While the iso-contours based on the tensor 
interpolation differ between interpolation, they have the same topology [b] and [c]. 
This is also reflected in the corresponding join trees, which are the same for both 
fields, for more details see Sect. 8. 

In the next step, we randomly perturb the directions of the tensors at the vertices 
without changing their eigenvalues to generate an ensemble of four tensor fields. Note 
that since we do not change the eigenvalues, the anisotropy at the vertices of the mesh 
remains unchanged after perturbation. Hence, for interpolation [a] all the fields, the 
iso-contours, the join trees, and their histograms will be the same as evident from 
Fig. 1 la. However, if we use quadratic function resulting form the tensor interpolation 
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[a] [b] [c] 


Fig. 10 Comparison of the behavior of the anisotropy field in a synthetic dataset. The anisotropy 
is linearly interpolated within the original mesh [a], and the subdivided monotonous triangles [b]. 
The anisotropy assuming linear interpolation of tensor components [c]. The upper row shows the 
resulting field as a color-map superimposed with contour lines. The ellipses at the vertices represent 
the tensors defining the field. The second row shows the respective join trees for the three anisotropy 
fields. The red squares correspond to the zeros inside of the original triangles. All zero-leaves in 
the tree represent degenerate points of the tensor field topology 


[c] of anisotropy, we clearly observe the differences within the ensemble members as 
shown in Fig. 11c. Similarly, the histograms for the ensemble members are different 
as shown by black curves in the plots in Fig. 11d. While for interpolation [b], the 
subdivision into monotonous triangles helps in identifying the differences, Fig. 1 1b, 
the histograms are still not accurate and have a bias toward larger anisotropy values, 
Fig. 11d. The respective contour trees are shown in Fig. 12. The contour tree for the 
interpolation [a] is the same for all 4 fields and already given in Fig. 11a, so we only 
show the trees for the sub-division in monotonous triangles. The degenerate points 
of the tensor field where the anisotropy is zero appear also as minima in the join 
tree and are highlighted in red. The join trees provide an overview of the possible 
cancellations of degenerate points and thus their stability [18]. Comparing the trees 
it can be seen that the four data sets vary significantly for their topological structure 
and stability of its degenerate points. The join tree for the linear anisotropy, Fig. | 1a, 
has no zero and is thus not consistent with the direction fields given by the tensors. 
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Histogram 


Histogram 


Histogram 


Fig. 11 Random directions tensor dataset with constant eigenvalues. a The original mesh with 
anisotropy computed at the mesh vertices and linearly interpolated within the triangles. b The sub- 
divided mesh with monotonous triangles. Anisotropy is linearly interpolated within the monotonous 
triangles. c The anisotropy resulting from linear interpolation of tensor components. d summarizes 
all histograms with red curves for interpolation [a], green curves for interpolation [b] and black 
curves for interpolation [c] 


8.2 Simulation Data 


The difference between the three different interpolations for the anisotropy field 
gave significantly different results for the synthetic data. In the next step, we want to 
investigate the impact of these differences on real-world data. At first, we consider a 
simulation data set from mechanical engineering. It is a well-known data set, often 
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Fig. 12 The join trees for the four ensemble members shown in Fig. 11. The trees summarize the 
hierarchical structure of the minima in the anisotropy field and the degenerate points in the tensor 
field topology. Degenerate points inside the triangles are marked as red nodes in the tree 


(a) (b) 


Fig. 13 Two-point load data: simulation of the stresses inside a solid block upon applying two 
forces. a shows a schematic of the set-up of the simulation, b shows the eigenvector directions in 
one slice, ¢ is a close up with color showing the signed anisotropy using bi-linear interpolation on 
the original mesh 


referred to as two-point load, that represents a stress field in a solid block resulting 
from the application of two external forces, Fig. 13. The data is given on a cubic 
mesh. The anisotropy measure used so far corresponds to the squared von Mises 
stress, which plays an important role in failure analysis of mechanical components. 
The direction field in one slice is shown in Fig. 13b. In our analysis, we consider 
a section of this slice, which is shown in Fig. 13c displaying one eigenvector field. 
Color represents the anisotropy using a bi-linear interpolation in the original mesh. 
In our analysis, we use a triangulated version of the data. 

Figure 14 shows the anisotropy fields using the three different interpolations in 
comparison. In Fig. 14a one can observe the asymmetry introduced by the triangu- 
lation but is similar to Fig. 13c. The anisotropy fields in Fig. 14b, c are both based 
on the monotonous subdivision and are very similar to each other but differ strongly 
from Fig. 14a. The asymmetry due to the triangulation is largely reduced. The min- 
ima inside the original triangles in the field capture the locations of the degenerate 
points of the direction field. The corresponding histogram can be seen in Fig. 14d, e. 
As expected they differ strongly for very small values but are rather similar for large 
values of the anisotropy. 
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Anisotropy i Anisotropy 
(d) (e) 


Fig. 14 Two point load simulation data with focus region around one of the loading points. a The 
original mesh with anisotropy computed at the mesh vertices and linearly interpolated within the 
triangles. b The subdivided mesh with monotonous triangles. Anisotropy is linearly interpolated 
within the monotonous triangles. c The anisotropy resulting from linear interpolation of tensor 
components. The isotropic points in the tensor field are highlighted by small red spheres. d the 
three cumulative histograms and (e) the histograms. The red curves correspond to (a), the green 
curves to (b) and the black curve to (ce) 


8.3 Measurement Data 


As the last example, we examine two sections of a slice of 3D Diffusion Tensor 
Imaging data. Specifically, we compute and compare the anisotropy histograms of 
noisy regions outside the brain and a region inside the brain. The 2D slice from the 
data along with selected regions is shown in Fig. 15. 

Figure 16 shows the histograms computed for the noisy region. With the interpo- 
lation approach [c] for computing histograms, we can observe a high contribution of 
anisotropy values near zero, which hints at the existence of a lot of degenerate points 
in the noisy region. This is not captured by the histogram computed with interpo- 
lation approach [a]. It should be noted that interpolation [b] (green plot) although 
yields better results than interpolation [a] (red plot), it is still quite different than the 
accurate histogram based on component-wise tensor interpolation (black plot). We 
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Fig. 15 Two segments selected from Diffusion Tensor Imaging data. The 20 x 20 2D grid shown 
in red outline is chosen from noisy region of the data while the selection shown in black outline is 
chosen from the region occupied by the brain. We plot the histograms for these selections in Figs. 
16 and 17 respectively 


did the same experiment for a region that is within the brain and hence contains less 
noise. The results are shown in Fig. 17. Here we observe, that histograms are very 
similar whether we use the accurate quadratic function for anisotropy of linearly 
interpolate it within the triangles. 


9 Conclusions 


In this paper, we explore the behavior of the anisotropy, as an example for a nonlinear 
derived tensor invariant, when applying a linear component-wise interpolation of the 
tensor field. We demonstrate that a linear interpolation of the invariant itself, the 
interpolation approach [a], leads to a topology of iso-contours that in many cases 
is not consistent with any tensor interpolation and leads to a bias in the histogram. 
With this analysis, we want to emphasize the importance of being consistent with 
the chosen interpolation in all analysis steps. For our analysis, we have chosen a 
linear interpolation of tensor components, which is the most commonly used method 
in simulations and provides a valid continuous field. We do not want to make a 
statement about the quality or suitability of different interpolation methods in specific 
applications, however we want to point out that the independent interpolation of 
the direction field and the anisotropy violates in many cases the preservation of 
topological invariants and does not result in a valid tensor field. 
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Cumulative Hist: Histogram 


Anisotropy Anisotropy 
(d) (e) 


Fig. 16 Anisotropy fields in a small noisy subset of brain dataset. a Linear interpolation in the 
original mesh. b Linear interpolation within the subdivided mesh with monotonous triangles. c 
The anisotropy resulting from linear interpolation of tensor components. d The comparison of 
cumulative histograms under these settings. e The comparison of the corresponding histograms 
with red (a), green (b), and black (c) 


More specifically we have presented a derivation for the computation of iso- 
contours and histograms in this setting. Component-wise linear interpolation of ten- 
sor components results in a quadratic function for anisotropy. The method is based 
on a subdivision of the mesh into triangles with monotonous behavior. This subdivi- 
sion with a linear interpolation of the anisotropy, interpolation approach [b], already 
results in iso-contours that are topologically equivalent to the iso-contours based on 
the derived anisotropy field. However, the histograms are not accurate and show a bias 
towards larger anisotropy values. This is especially prominent in regions with many 
degenerate points. In areas of high anisotropy, interpolation [b] provides a good 
approximation. The method described in this chapter, the interpolation approach 
[c], can be used to compute an accurate continuous histogram for anisotropy using 
component-wise tensor interpolation. 
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Cumulative Histogram Histogram 


Anisotropy Anisotropy 
(d) (e) 


Fig.17 A small subset of DTI brain dataset is selected within the brain region. a Linear interpolation 
in the original mesh, b Linear interpolation in monotonous triangles. c The anisotropy resulting from 
linear interpolation of tensor components. d and e display the respective cumulative histograms and 
histograms as (a) red, (b) green, (c) black plots 


All derivations in this chapter are given for the anisotropy defined as the difference 
between the major and minor eigenvalue, for 2D tensor fields. Although not trivial, the 
extension of this work to 3D tensor fields is feasible. An extension to the determinant, 
which is also quadratic but not elliptic, would be possible in a similar way. A general 
extension to other non-linear tensor invariants, however, might not be possible in 
a closed form and will require a good approximation schema. Therefore, we plan 
to explore methods for efficient approximations to the distributions with clear error 
bounds. Also, computing continuous scatterplots to visualize the space of multiple 
invariants at once is an interesting topic and will be subject of future work. 
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Appendix A 


A. Detailed Analysis of Anisotropy and Its Iso-Contours 
Proof of H > 0 


Substituting the Eqs. (14)—(19) in Eq. (6), we can analyse the determinant of Hessian 
for anisotropy v: 
H =4AC — B’ 
=4((ex — 81)? + 4F2)((ey — By)? +42) — 4((ex — Bx) (ey — By) HARRY 
=4((ex — 82) (ey — By) +44 (ey — By) +46 ex — 8x) + 16K, fF) 
— 4((ex = gx)" (ey — By)” + 8 fx fy (Cx — Bx)(Cy — By) + 16f? fy) 
=16( f? (ey — 8y) + fy (ex — 8x)” — 2 fr fy(€x — 8x) (ey — 8y)) 
H =16(f;, (ey — 8y) — fyl€x — 8) = 0 (30) 


Since H > 0, we have shown that the iso-contours of vare never hyperbolic. 


Proof of I = 0 when H = 0 


Let us analyze the case when H = 0 to complete the analysis of behaviour of 
anisotropy in all cases. 


H =16(f, (ey = gy) = fy (ex = gy) = (0) 
or, Sc (ey = gy) = Sy (ex =g) =0 


or, 2 = By — 2 
ey — 8x x 
ty 

or, ey— By = 7e — gx) (31) 


Substituting (31) in (16) and using (14): 


C = ((ey — By) + 4f,) 
= La = &x)° 


a 
2 
= (ce = 8x) + 4/2) 
3 
or, C= 2A (32) 


E 
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Substituting (31) in (18) and using (17): 


E = 2((ey = 8y)(ec — 8e) + 4 fy fe) 


= o( 2% ie = gc) +A) 


fy 
= F i 2((ex 8x) (ec — Bc) + 4 fx fe) 
or, E= typ (33) 
fc 
From Eqs. (32) and (33): 
C FE ‘ 
== 2, (34) 
A D2 f2 
or, AE? = CD? (35) 
Substituting (31) in (15) and using (16): 
B = 2((ex — 8x)(ey — 8y) + 4 fx fy) 
g3 
= (26 gy) + AfA) 
fy 
= 22 (« Aa 4/2) 
ty 
or, B= atte (36) 
fy 
Multiplying Eqs. (33) and (36), we obtain: 
BE=2CD (37) 
Let us evaluate the Eq. 7 now: 
I = BDE — AE? — CD’ 
I = BDE -2C D? using (35) 
I = D(BE — 2CD) 
I= D(O)=0 using (37) (38) 


From Eq. (38), we conclude that when H is 0, J is also 0. This means that the 
iso-contours of v are never parabolic. 

To conclude, based on Eqs. (30) and (38), we deduce that the contours of 
anisotropy v and hence a are either ellipses or a set of parallel lines. 
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Abstract Tensor decomposition methods and multilinear algebra are powerful tools 
to cope with challenges around multidimensional and multivariate data in computer 
graphics, image processing and data visualization, in particular with respect to com- 
pact representation and processing of increasingly large-scale data sets. Initially 
proposed as an extension of the concept of matrix rank for 3 and more dimensions, 
tensor decomposition methods have found applications in a remarkably wide range 
of disciplines. We briefly review the main concepts of tensor decompositions and 
their application to multidimensional visual data. Furthermore, we will include a first 
outlook on porting these techniques to multivariate data such as vector and tensor 
fields. 


1 Introduction 


Data approximation is widely used in the fields of computer graphics and scientific 
visualizations. One way to achieve it is to decompose the data into a more compact 
and compressed representation. The general idea is to express a dataset by a set of 
bases, which are used to reconstruct the dataset to its approximation when needed (see 
Fig. 1). More specifically, such a representation usually consists of the actual bases 
and the corresponding coefficients describing the relationship between the original 
data and the actual bases. Typically, such compact representations consist of less 
data than the original dataset, capture the most significant features, and, moreover, 
describe the data in a format that is convenient for adaptive data loading and access. 
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compact data representation 


> bases + 


decompose coefficients reconstruct 


Fig. 1 Compact data representation for a 3rd-order tensor A (a volume) by bases and coefficients 
that can be used to reconstruct the data to its approximation A 


Bases for compact data representation can be classified into two types: pre-defined 
and learned bases. Pre-defined bases comprise a given function or filter, which is 
applied to the dataset without any a-priori knowledge of the correlations in the dataset. 
In contrast, learned bases are generated from the dataset itself. Established examples 
of the former are the Fourier transform (FT) and the wavelet transform (WT). Well- 
known examples of the latter are the principal component analysis (PCA), the singular 
value decomposition (SVD) and vector quantization. Using predefined bases is often 
computationally cheaper, but learned bases potentially remove more redundancy 
from a dataset. 

Generally, PCA-like methods are able to extract the main data direction of the 
dataset and represent the data in another coordinate system such that it makes it 
easier for the user to find the major contributions within the dataset. To exploit 
this, PCAs higher-order extension—tensor approximation (TA)—can be used for 
multidimensional datasets. The first occurrence of TA was in [21]. The idea of multi- 
way analysis, however, is generally attributed to Catell in 1944 [11]. It took a few 
decades until tensor approximations received more widespread attention, namely by 
several authors in the field of psychometrics [10, 19, 52]. 


1.1 Higher-Order Data Decompositions 


The matrix SVD works on 2D data arrays and exploits the fact that the dataset can 
be represented with a few highly significant coefficients and corresponding recon- 
struction vectors based on the matrix rank reduction concept. The SVD, being a 
multilinear algebra tool, computes (a) a rank-R decomposition and (b) orthonormal 
row and column vector matrices. Unlike for 2D, the extension to higher-orders is not 
unique and these two main properties are captured by two different models that are 
both called tensor approximations : the Tucker model [14, 15, 25, 49, 52] preserves 
the orthonormal factor matrices while the CP model (from CANDECOMP [10] and 
PARAFAC [19]) preserves the rank-R decomposition. 

Generally speaking, a tensor is a term for a higher-order generalization of a vector 
or a multidimensional array. In TA approaches, a multidimensional input dataset in 
array form, i.e., a tensor, is factorized into a sum of rank-one tensors or into a product 
of a core tensor (coefficients that describe the relationship to input data) and matrices 
(bases), i.e., one for each dimension. This factorization process is generally known 
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as tensor decomposition, while the reverse process of the decomposition is the tensor 
reconstruction. 

Tensor decompositions have been widely studied in other fields and were reviewed 
[13, 25, 34] and summarized [27, 44]. Since TA was emerging from various disci- 
plines, it was developed under multiple names. The CP model was independently 
developed under the terms CANDECOMP and PARAFAC, therefore, it is some- 
times referenced with a single name. The Tucker model takes its name from Tucker, 
who initially worked on the three-mode factor analysis (3MFA), which is sometimes 
referred to as the Tucker3 model, also called the three mode PCA (3MPCA) [27, 28, 
49]. Similarly the model was referenced to as n-mode PCA [23] since it is equivalent 
to applying PCA n times, each time along a different mode of the tensor. In [14] 
all these previous works were captured and written down as generalization of the 
SVD as multilinear singular SVD, which is usually termed as higher-order SVD or 
HOSVD. Furthermore, it was also called n-mode SVD in [54, 55]. 

Given an input tensor, different decompositions capture different types of struc- 
tures and result in varying numbers of coefficients. For a given accuracy, the number 
of CP ranks required to decompose a tensor is usually much larger to that of Tucker 
ranks. On the other hand, CP’s storage cost grows only linearly with respect to its 
ranks, whereas that relationship becomes exponential in the case of Tucker. In sum, 
there is no silver bullet: CP is more suitable than Tucker for certain types of data, 
and vice versa. As a rule of thumb: 


e Dense tensors of moderate dimensionality n over continuous variables, for example 
n = 3 0rn = 4 including spatial and temporal axes, can often be compressed more 
compactly via the Tucker model. 

e Tensors with categorical variables, sparse tensors, and tensors of higher dimen- 
sionality (say, n > 5) often benefit more from the CP model. 


The data sets addressed in this chapter fit in the first category, and so we restrict our 
experiments to the Tucker model. To further illustrate the usual advantage of Tucker 
over CP for spatial visual data, see Fig. 2, we compare the number of coefficients 
and root mean squared error (RMSE) obtained with CP vs. Tucker using different 
numbers of ranks for a 256 x 256 x 256 CT scan of a bonsai. 


1.2 TA Applications in Graphics and Visualization 


TA approaches have been applied to a wide range of application domains. Starting 
from psychometrics, in recent years, TA has been applied to visual data. A highly 
studied area is TA used for image ensembles [20, 35, 42, 43, 54, 58-60, 63] and/or 
TA used for pattern recognition, e.g., [17, 32, 40, 41, 43, 59]. In (real-time) ren- 
dering, tensor decompositions have been used as a method for global illumination 
models, e.g., for bidirectional reflectance distribution functions (BRDFs) [9, 45] or 
precomputed radiance transfer (PRT) [45, 50, 51]. Furthermore, TAs have success- 
fully been used in graphics in the context of bidirectional texture functions (BTFs) [3, 
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Fig. 2, RMSE achieved after CP vs. Tucker compression at different ranks R. For Tucker we take 
R, = Ro = R3 = R. The dataset used is a bonsai CT scan of size 256° 


18, 38, 39, 51, 55, 61, 62], texture synthesis [62], time-varying visual data [61, 62], 
3D face scanning [56] and compression in animation [29, 31, 33, 36, 37, 53, 57]. 

In scientific visualization, TA methods have first been introduced for interactive 
multiresolution and multiscale direct volume rendering [6—8, 46—48]. Additionally, 
their compact representation power has been exploited for 3D volume data compres- 
sion [4, 6] with notable advantages over other approaches at extreme compression 
ratios [2]. In this work, we explore the multiscale feature expressiveness of TA meth- 
ods for the first time on vector fields, i.e. multidimensional multivariate data . Hence 
we interpret the vector field as a 4D array, or as a 4th-order data tensor. However, 
the rank-reduction of TA is only applied to the three spatial dimension. 


1.3 Motivation and Contributions 


In the results section (Sect. 9) we demonstrate that the feature sensitive approxi- 
mation power of TA methods carries over from scalar to vector fields. These first 
results are promising and encourage the extension of tensor compression techniques 
to multivariate data fields, e.g. for compact storage and quick visualization at variable 
feature scales of large vector data in the fields of computational fluid dynamics or 
biomedicine. Moreover, based on the compression-domain data filtering and process- 
ing capabilities demonstrated in [5], we expect that important vector-field operators 
such as divergence or vorticity as well as other feature extraction operations can be 
analyzed and performed directly in the compressed TA format. 
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2 Singular Value Decomposition 


The SVD is a widely used matrix factorization method to solve linear least-square 
problems. The SVD can be applied to any square or rectangular matrix A € RY”*%, 
Hence, the decomposition is always possible. The aim of the SVD is to produce a 
diagonalization of the input matrix A. Since the input matrix A is not symmetric, two 
bases (matrices) are needed to diagonalize A. Therefore, the SVD produces a matrix 
factorization into two orthogonal bases U e R“*™ and V e RY*™ and a diagonal 
matrix © € R“*", as expressed in Eq. (1) (matrix form) or Eq. (2) (summation 
form). 


A= UV" d) 
R 
Amn = D UmrOr Unr (2) 


r=1 


The bases U and V contain orthogonal unit length vectors u; and v}, respectively, 
and represent a r-dimensional column space (IR™) and a r-dimensional row space 
(R™). Hence, the bases U and V are even orthonormal. The diagonal matrix © con- 
tains the singular values o,, where o; > 02 > --- > Or > 0, where R = min(M, N). 
A singular value and a pair of singular vectors of a square or rectangular matrix A 
correspond to a non-negative scalar ø and two non-zero vectors u; and vj, respec- 
tively. The vectors u; are the left singular vectors, and the vectors v; are the right 
singular vectors (see Fig. 3). The number of non-zero singular values determines the 
rank R of the matrix A. 

In applications truncated versions of the SVD are frequently desired. That is, 
only the first K singular values o; . .. og and the corresponding K singular vectors 
u; ... Ug and vı ... Vx are used for the reconstruction. This approach is referred to as 
low-rank approximation of a truncated SVD. Basically, each weighted outer vector- 
product term g; - uj o v; corresponds to a rank-one component (see also Fig. 4), and 
the SVD of matrices or images consequently represents a 2D data array eventually 
as a sum of such rank-one components. 


BJE CoD || | | ) 
vı v2 VR 
lam + ib 
u u tot UR 


Fig. 3 Visualization of the summed form of the SVD as shown in Eq. (2)—illustrating the singular 
values with the corresponding left and right singular vector pairs 
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Fig. 4 Simple 2D functions can be represented as an outer product of two 1D functions, and more 
complex ones as a weighted sum of several such components 


3 Tensor Approximation Notation and Definitions 


The notation taken here is mostly taken from De Lathauwer et al. [13, 14], Smilde 
et al. [44], as well as Kolda and Bader [25], who follow the notation proposed by 
Kiers [24]. To illustrate higher-order extensions we mostly make examples of order 
three. 


3.1 General Notation 


A tensor is a multidimensional array (or an N-way data array): a Oth-order tensor 
(tensor0) is a scalar, a 1st-order tensor (tensor1) is a vector, a 2nd-order tensor is a 
matrix, and a 3rd-order (tensor3) is a volume, see Fig. 5. We consistently use the 
letter ‘a’ to represent the data, following the notation of, e.g., [14, 15, 51, 61, 62]. 
We use lower case letters for a scalar a, lower case boldface letters for a vector a 
in R”, capital boldface letters for a matrix A in R"*”, and calligraphic letters for a 
3rd-order tensor A in R'*”2*4, 


Fig.5 A tensor is a multidimensional array: a 2nd-order tensor is a matrix A and a 3rd-order tensor 
is a volume A 
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(a) frontal A;, (b) horizontal Aj, (c) lateral A; 


Fig. 6 Slices of a 3rd-order tensor A 


The order of a tensor is the number of data directions, also referred as ways or 
modes. Along a mode n, the index i, runs from 1 to Iy. By using lower script indices 
for the modes, we can extend the index scheme to any order, i.e., 1, D, 3, ly,.... 
The ith entry of a vector a is denoted by a;, an element (i), i2) of a matrix A is 
denoted by a;,;,, and an element (i1, i2, i3) of a 3rd-order tensor A is denoted by 
iy iniz- 

The general term fibers is used as a generalization for vectors taken along different 
modes in a tensor. A fiber is defined by fixing every index but one. A matrix column 
is thus a mode-1 fiber and a matrix row is a mode-?2 fiber. 3rd-order tensors have 
column, row, and tube fibers, denoted by a;,, a;,, and a;,, respectively. Sometimes, 
fibers are also called mode-n vectors. 

Slices are two-dimensional sub-sections of a tensor (e.g., one fixed index in a 3rd- 
order tensor). For a 3rd-order tensor A, there are, for example, frontal, horizontal, 
and lateral slices, denoted by A;,, Aj,, and A;,, respectively as illustrated in Fig. 6. 

For computations, a tensor is often reorganized into a matrix what we denote 
as tensor unfolding (sometimes called matricization). There are two main unfold- 
ing strategies, backward cyclic unfolding [14] and forward cyclic unfolding [24] as 
shown in Fig. 7. An unfolded tensor in matrix shape is denoted with a subscript in 
parentheses, e.g., Aqn). 


3.2 Computing with Tensors 


While many more operations on tensors exist, here we only outline the most common 
products used within the scope of this work. 


e An Nth-order tensor is defined as A € R'*2%" IN, 

e The tensor product is denoted here by ®: however, other symbols are used in the 
literature, too. For rank-one tensors, the tensor product corresponds to the vector 
outer product (o) of N vectors b™ e R” and results in an Nth-order tensor A. The 
tensor product or vector outer product for a 3rd-order rank-one tensor is illustrated 
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Fig. 7 Forward cyclic 
unfolding [24] of a 3rd-order A 
tensor 
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in Fig. 8: A= b® o b® o b®), where an element (i1, i2, i3) of A is given by 
24,2) 

Gijinis = iy “iy 7%i3 * 

The inner product of two same-sized tensors A, B € R2 =XIn is the sum of 

the products of their entries, i.e., Eq. (3). 


(ABS yy ia Sic: in Pinin..in (3) 


i= lin= 1 in=l 


The n-mode product [14] multiplies a tensor by a matrix (or vector) in mode n. 
The n-mode product of a tensor B e R'*2*"*! with a matrix C € R**™ is 
denoted by 8 x,, C and is of size I) x +--+ x In-1 X Jn X Ings X +++ X Iy. That 
is, element-wise we have Eq. (4). 


A 
(8 Xn C)iisinijiingi iy = X Dinin.in * Cinin (4) 


i=l 


Each mode-n fiber is multiplied by the matrix C. The idea can also be expressed 
in terms of unfolded tensors (reorganization of a tensor into a matrix as described 
in Sect. 3.1). 

The n-mode product of a tensor with a matrix is related to a change of basis in 
the case when a tensor defines a multilinear operator [25]. The n-mode product is 
the generalized operand to compute tensor times matrix (TTM) multiplications, 
and can best be illustrated using unfolded tensors as in Fig. 9. 
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Fig. 8 Three-way outer Po 
product for a 3rd-order 
rank-one tensor = n 
A= b ob® o b® l = b 
bD 
I 
h : 
Fig. 9 TTM multiplication ih 
C- Bin), multiplying the B 1 I 
unfolded 3rd-order tensor (n) | 1 ig 
Ban) with the matrix C 
" fot i | 
In ih 
A=Bx,Cs Aw) = CB) (5) 


e The norm ofa tensor A € R"*R*xIn is defined analogously to the matrix Frobe- 
nius norm ||A||- and is the square root of the sum squares of all its elements, i.e., 


Eq. (6). 


qi h Iy 
lAlr= |S ee (6) 


i=l i=1 in=1 


3.3 Rank of a Tensor 


In order to describe the definitions of the tensor rank, the definition for the matrix 
rank is recaptured. The matrix rank of a matrix A is defined over its column and row 
ranks, i.e., the column and row matrix rank of a matrix A is the maximal number 
of linearly independent columns and rows of A that can be chosen, respectively. For 
matrices, the column rank and the row rank are always equal and, a matrix rank is 
therefore simply denoted as rank(A). A tensor rank is defined similarly to the matrix 
rank, however, there are differences. In fact, the extension of the rank concept is not 
uniquely defined in higher orders and we review the definitions for the tensor ranks 
from [14] here. 


e The n-rank of a tensor A, denoted by R, = rank, (A), is the dimension of the 
vector space spanned by mode-n vectors, where the mode-n vectors of A are 
the column vectors of the unfolding An), and rank, (A) = rank(A(,)). Unlike 
matrices, the different n-ranks of a tensor are not necessarily the same. 
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e A higher-order tensor has a so called multilinear rank (R1, R2, ..., Ry) [21] if its 
mode-1 rank (row vectors), mode-2 rank (column vectors) until its mode-n rank 
are equal to Ri, Ro,..., Ry, e.g., giving rise to a multilinear rank-(R;, R2, R3) 
for a 3rd-order tensor. 

e A rank-one tensor is an N-way tensor A € R!*”*"*!" under the condition that 
it can be expressed as the outer product of N vectors, as in Eq. (7) (see also [12, 
30]). A rank-one tensor is also known under the term Kruskal tensor. 


A =b” ob” 0... o b™ (7) 


e The tensor rank R = rank(A) is the minimal number of rank-one tensors that 
yield A in a linear combination (see [12, 14, 25, 30]). Except for the special case 
of matrices, the tensor rank is not necessarily equal to any of its n-ranks, but it 
always holds that R, < R. 


4 Tensor Decompositions 


In tensor decompositions an input tensor A € R!*”*~*!" is decomposed into a set 
of factor matrices U™ e R!** and coefficients B e RF:*R2*=xRN that describe 
the relationship/interactivity between A and the set of U™. 

Historically, as seen earlier, tensor decompositions are a higher-order extension 
of the matrix SVD. The nice properties of the matrix SVD, i.e., rank-R decompo- 
sition and orthonormal row-space vectors and column-space vectors, do not extend 
uniquely to higher orders. The rank-R decomposition can be achieved with the so- 
called CP model, while the orthonormal row and column vectors are preserved in the 
so-called Tucker model. An extensive review of the two models and further hybrid 
models can be found in [25]. Here, we only outline the Tucker model that we apply 
in our experiments. 


4.1 Tucker Model 


The Tucker model is a widely used approach for tensor decompositions. As given 
in Eq. (8), any higher-order tensor is approximated by a product of a core tensor 
Be RM xRx-*Rw and its factor matrices U™ e R”**:, where the products x, 
denote the n-mode product as outlined in Sect. 3.2. This decomposition can then 
again be reconstructed to its approximation A. The missing information of the input 
tensor A that cannot be captured by A is denoted with the error e. The Tucker 
decomposition is visualized for a 3rd-order tensor in Fig. 10. Equivalently, a Tucker 
decomposition can also be represented as a sum of rank-one tensors as in Eq. (9) and 
illustrated in Fig. 11. 
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Fig. 10 Tucker 3rd-order tensor: A = 8 x UM x2 U® X3 U® +e 


3 
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Fig. 11 Tucker 3rd-order tensor as a sum of rank-one tensors: A = wad 1 yi 


ul o u? o u® +e 


r2=1 R= bau? 


A = B xı UV x, UP x3--- xy UM +e (8) 
Rı Ro 

A= yoy oe ry WD ou o---ou™ $e O) 
ry=ln=1 ry=l 


The column vectors u) of the factor matrices U™ e R™*® are usually orthonor- 
mal and can be thought of as principal components R, in each mode n [25]. 
The core tensor B e RFI*R:*=xRN represents a projection of the original data 
A e R'*2*~~*Iy onto its factor matrices and is always of the same order as the 
input data. The core tensor is computed in general, as shown in Eq. (10), and for 
orthogonal factor matrices as in Eq. (11). The element-wise core tensor computation 
is denoted in Eq. (12). In other words, the core tensor coefficients b, r,...ry represent 
the relationship between the Tucker model and the original data. 


B = A x4 UO x, VIM x3. xy UME? (10) 


B=A x, UO" x UP! xz- xy UVT (11) 


hi h 


T 2) T N) T 
8=) 2) Sa iy Uy) ou,” orou (12) 


1j=1 in=1 iy=1 


The Tucker decomposition is not unique, which means that we can modify the 
core tensor B without affecting the model fit as long as we apply the same changes 
to the factor matrices (so-called core tensor rotations), for more details see [25]. 
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Often, we are interested in compact models, which enable a compression of the 
input data. For example, after computing a (full) Tucker decomposition the core 
tensor $ has the same size as the original input A and all the factor matrices are 
square. However, we are more interested in reduced-size, approximative Tucker 
decompositions, where 8 is an element of R*'**2*®3 with Ry < I, Ry < h and 
R3 < 13. Using so-called rank-reduced tensor decompositions or truncated tensor 
decompositions one can directly obtain more compact decompositions. 


5 Tensor Rank Reduction 


As seen in Sect. 3.3, the extension of the matrix rank concept to higher orders is not 
unique and we will mostly follow the rank-(R), Ro, ..., Ry) tensor decomposition 
and reduced-rank approximation of the Tucker model here. 


5.1 Rank-R and Rank-(R,, R2, ..., Rn) Approximations 


A simple rank-one approximation is defined as A = 1 - u® o u® -. - o u™) from the 
rank-one tensor (vector) product (0) of its N basis vectors u™ € R™ and a weight 
factor A. Hence a tensor A could be approximated by a linear combination of many 
rank-one approximations as in Eq. (13). This approximation, also known as a CP 
model, is called a rank-R approximation. 


R 
A= a, uP ou o.. ou (13) 


Poneman if we allow all weighted tensor (vector) products uso u® o -o 


u” ) of any arbitrary index combinations i172... iy, we end up with the Tucker model 
of Sect. 4.1 and Eq. 12 where the weight factors for all index combinations form the 
core tensor $. Choosing Rj... y we end up with a rank-(Rj, Ro, ..., Ry) 
approximation of A, which is given by a decomposition into a lower-rank tensor 
Ae RXXX with rank, (A) = Rn, < rank, (A). The approximated tensor A 
is the n-mode product x, of factor matrices U™® e R'**« and a core tensor B € 
IRRix Rox Rw in a given reduced rank space (Eq. (14)). This rank-(R,, R2, ..., Ry) 
approximation was previously introduced as the Tucker model. 


arias 


A = B xı UV x, UË x3- xy UM (14) 


In general, a rank-reduced approximation is sought such that the least-squares 
cost function of Eq. (15) is minimized. 
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A = arg min |A — All * (15) 
A 


Given that (Ri, Ro,..., Ry) are sufficiently smaller than the initial dimensions 
(L, In,..., Iy), the core coefficients B e R***2**®y and the factor matrices 
U™ © R&*%» can lead to a compact approximation of A of the original tensor A. 
In particular, the multilinear rank-(R), Ro, ..., Ry) is typically explicitly chosen to 
be smaller than the initial ranks in order to achieve a compression of the input data 
(see also [2, 4, 6]). 


5.2 Truncated Tensor Decomposition 


Similar to matrix SVD, tensor rank reduction can be used to generate lower-rank 
reconstructions A of the input A. The tensor rank parameters R,, are chiefly respon- 
sible for the number of TA bases and coefficients that are used for the reconstruction 
and hence are responsible for the approximation level. In higher orders, the CP 
decomposition is not directly rank-reducible, however, the truncation of the Tucker 
decomposition is possible due to the all-orthogonality property of the core tensor. 

For a 3rd-order tensor, all-orthogonality means that the different horizontal matrix 
slices of the core $ (the first index 7; is kept fixed, while the two other indices, iz and 
i3, are free) are mutually orthogonal with respect to the scalar product of matrices 
(i.e., the sum of the products of the corresponding entries vanishes). The same holds 
for the other slices with fixed indices iz and iz (see [14]). Therefore, given an initial 
sufficiently accurate rank-(R,, R2, R3) Tucker model, we can progressively choose 
lower ranks K„ < R, for reduced quality reconstructions. As indicated in Fig. 12, the 
ranks K, indicate how many factor matrix columns and corresponding core tensor 
entries are used for the reconstruction. 

Note that the ordering of the coefficients in the Tucker core tensor $ is not strictly 
decreasing in contrast to the decreasing singular values in the matrix SVD case. 
However, in practice it can be shown that progressive tensor rank reduction in the 
Tucker model works very well for adaptive reconstruction of the data at different 
accuracy levels. 


Fig. 12 Illustration of a rank reduced Tucker tensor reconstruction: A reduced range of factor matrix 
columns with corresponding fewer core tensor entries reconstructs a lower quality approximation 
but at full resolution 
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6 Tucker Decomposition Algorithms 


There are various strategies for how to compute and generate a tensor decomposition. 
The most popular and widely used group of algorithms belongs to the alternating 
least squares (ALS) algorithms, the other group of algorithms uses various Newton 
methods. The respective algorithms differ for the computation of the different tensor 
models, and we will mainly focus on the Tucker model in our review. 

For the Tucker model, the first decomposition algorithms used were a simple 
higher-order SVD (HOSVD) (see [14]), the so-called Tucker] [52], a three-mode 
SVD. However, the truncated decompositions of higher orders are not optimal in 
terms of best fit, which is measured by the Frobenius norm of the difference. Starting 
from a HOSVD algorithm, tensor approximation ALS algorithms [26, 28] were 
developed, where one of the first Tucker ALS was the so-called TUCKALS [49]. Later 
various improvements accelerated [1] or optimized the basic TUCKALS method. The 
higher-order orthogonal iteration (HOOI) algorithm [15] is an iterative algorithm 
that performs a better fit for a truncated HOSVD version. 

Newton methods are also used for the Tucker decomposition or rank-(R,, Ro, ..., 
Ry) approximation. They typically start with a HOOI initialization and then converge 
faster to the final point. [16] developed a Newton-Grassman optimization approach, 
which takes much fewer iterations than the basic HOOI - even though one single iter- 
ation is more expensive due to the computation of the Hessian. While the HOOT is not 
guaranteed to converge, the Newton-Grassmann Tucker decomposition is guaranteed 
to converge to a stationary point. Another Newton method was proposed by [22], 
who developed a differential-geometric Newton algorithm with a fast quadratic con- 
vergence of the algorithm in a neighborhood of the solution. Since this method is 
not guaranteed to converge to a global maximum, they support the method by start- 
ing with an initial guess of several HOOI iterations, which increases the chances of 
converging to a solution. 


7 Tensor Reconstruction 


The tensor reconstruction from a reduced-rank tensor decomposition can be achieved 
in multiple ways. One alternative is a progressive reconstruction: Each entry in the 
core tensor B is considered as weight for the outer product between the corresponding 
column vectors in the factor matrices U™ . This gives rise to Eq. (16) for the Tucker 
reconstruction. 


Ri Ro Ry 
~ , ; p 
A= BP ae ; Driry.rN su? ou o... oul (16) 
r=lr=l ry=1 


This reconstruction strategy corresponds to forming rank-one tensors and cumu- 
latively summing them up. The accumulated weighted subtensors then form the 
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approximation A of the original data A. In particular for the Tucker model, this is 
an expensive reconstruction strategy since it involves multiple for-loops to run over 
all the summations, for a total cost of O(R™ - I”) operations. 


7.1 Element-Wise Reconstruction 


A simple approach is to reconstruct each required element of the approximated 
dataset individually, which we call element-wise reconstruction. Each element Gj, izis 
is reconstructed, as shown in Eq. (17) for the Tucker reconstruction. That is, all core 
coefficients multiplied with the corresponding coefficients in the factor matrices are 
summed up (weighted sum). 


im 1 2 N 
iini = YO brrr Un ey ee (17) 


inr2 inn 
Fi tN 


Element-wise reconstruction requires O(R™) operations on average. It can be 
beneficial for applications where only a sparse set of reconstructed elements are 
needed. 


7.2 Optimized Tucker Reconstruction 


A third reconstruction approach—applying only to the Tucker reconstruction—is to 
build the n-mode products along every mode, which leads to a TTM multiplication 
for each mode, e.g., TTM1 along mode 1, (see also Eq. (5)). This is analogous to 
the Tucker model given by Eq. (14). The intermediate results are then multiplied 
along the next modes, e.g., TTM2 and finally TTM3. In Fig. 13 we visualize the 
TTM reconstruction, and the intermediate results B’ and B” as well as the final 
approximation A, applied to a 3rd-order tensor using n-mode products. 


(a) TT™M 1 (b) TTM 2 (c) TTM 3 


Fig. 13 Forward cyclic TTM multiplications after [24] along the three modes (n-mode products) 
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Since it exploits matrix multiplications, this optimized algorithm is much faster 
than progressive reconstruction (Eq. (16)). Its cost is dominated by O(R, -I N), 
In 3D, in particular, it takes O(R, - I) - b+ 13+ Ry: R- h-hbh++R R- R3- Ih) 
operations. 


8 Useful TA Properties for Scientific Visualization 


As stated in the introduction, TA is the higher-order generalization of the matrix SVD, 
which can offer either properties of (a) rank-R decomposition or (b) orthonormal 
row-space and column-space vectors. In higher orders, the orthonormal row and 
column vectors are preserved in the Tucker model which thus supports progressive 
rank-reduced approximations and reconstructions. 


8.1 Spatial Selectivity and Subsampling 


The Tucker model (Sect. 4.1) consists of one factor matrix per mode (data direction) 
U™ e R"** and one core tensor B e R*'***-*8w The core tensor 8 is in effect 
a projection of the original data A onto the basis of the factor matrices U™. In case 
of a volume, the Tucker model has three modes, as illustrated in Fig. 10, and defines 
an approximation A = B x, U x, U® x3 U® of the original volume A (using 
n-mode products x,). 

The row and column axes of the factor matrices represent two different spaces: 
(1) the rows correspond to the spatial dimension in the corresponding mode, and 
(2) the columns to the approximation quality. These two properties can be exploited 
for multiresolution modeling (spatial selection and subsampling of rows) and mul- 
tiscale approximation (rank reduction on the columns) (see also Fig. 14). In [5, 47] 
we demonstrated how these features can be exploited for multiresolution and multi- 
scale reconstruction as well as filtering of compressed volume data in the context of 
interactive visualization. 


Fig. 14 Factor matrix 
properties along the vertical 
axis supporting: (1.1) spatial 
selectivity, (1.2) spatial 
subsampling, and (2) 
low-rank approximation 
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8.2 Approximation and Rank Reduction 


As described earlier, the Tucker model defines a rank-(R1, R2, R3) approximation, 
where a small R„ corresponds to a low-rank approximation (with details removed) 
and a large R, corresponds to a more accurate approximation of the original. The 
highest rank R,, for the initial Tucker decomposition has to be given explicitly. 
However, rank reductions can be applied after the initial decomposition (similar to 
the rank reduction in matrix SVD). Even though the core tensor coefficients are not 
guaranteed to be in decreasing order, as in matrix SVD, in practice it can be shown 
that progressive tensor rank reduction in the Tucker model works well for adaptive 
visualization of the data at different feature scales [2, 4—6, 47]. 

As illustrated in Fig. 12 in Sect. 5.2, the ranks indicate how many factor matrix 
columns and corresponding core tensor entries are used for a desired reconstruction. 
Thus, given a rank-(R;, R2, R3) Tucker model, we can specifically or progressively 
choose lower ranks K,, < R, for a reduced quality reconstruction, at the original 
spatial output resolution given by J, (or also subsampled). 

Figure 15 shows the progressive rank reduction from an initial rank-(256, 256, 
256) Tucker decomposition of an original 512° example volume. Shown are the 
visual results and the data reduction of the approximation at variable reduced-rank 
reconstructions. The numbers of coefficients include all core tensor and factor matrix 
entries that are used, e.g. a rank-(32, 32, 32) reconstruction corresponds to 327 +3. 
512 - 32 = 81/920 coefficients. The data reduction ratio can be derived by dividing 
the number of coefficients by 512°, which for R = 32 results in using only 0.06% 
of the original amount of data. In particular, Fig. 15 demonstrates the power of low- 
rank tensor approximations that can be used for multiscale feature visualization or 
progressive image refinement in volume rendering. 


oH 


28'672 81'920 360'448 2'293'760 17'170'432 
R=16 R = 32 R = 64 R = 128 R = 256 


number of coefficients 


Fig. 15 Multiscale volume visualization by tensor rank reduction, corresponding to 0.02, 0.06, 
0.27, 1.71 and 12.79% of the original amount of data used for R = 16, 32, 64, 128, 256 
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9 Application to Multivariate Data 


Encouraged by the data reduction power and the approximation quality of tensor 
approximations as shown in Fig. 16, we have extended the Tucker decomposition 
and reduced rank reconstruction to vector field data. Compared to scalar 3D volumes 
(e.g. from MRI or CT scans), a 3D vector field V (e.g. of velocities from a weather or 
fluid simulation) can be interpreted as a multivariate data field with D-dimensional 
vector-valued entries at each position ina J; x I, x J; grid. Thus we can interpret V 
as a 4th-order tensor V € R'*”*45*?_ Note that the TA rank-reduction (Sect. 5) is 
only applied to the three spatial dimension as we are not interested in a dimensionality 
reduction of the vector-valued field value itself. 


9.1 Dataset 


For our first experiments we used a data set from the Johns Hopkins Turbulence 
Database http://turbulence.pha.jhu.edu/ representing a direct numerical simulation 
of incompressible magnetohydrodynamic (MHD) equations (see also Fig. 17). The 
data set contains, among other output variables, 3 velocity components that we used, 
hence D = 3, covers a 3D grid of 256° cells (downsampled subset from the original 
10243), hence 1; = Ib = I, = 256, and is the first time step from the output of the 
simulation. 

The vector field therefore covers a 3D cube which we visualize using direct vol- 
ume and streamline rendering techniques with color-coding of velocity, divergence, 
vorticity or error magnitudes in ParaView. Transparency is used to reduce clutter 
and opacity such as to focus the visualizations on the high-magnitude value regions. 
Note that this cubic vector field is dense and rendered over a black background, thus 
always appearing as a cube like object in the images. 


(a) Original (b) 10:1 (c) 300:1 


Fig. 16 a A 5123 isotropic turbulence volume of 512MB, b visually identical compression result 
using 51.2MB and ¢ result after extreme compression down to only 1.71MB using the TTHRESH 
method [2]. All the visualizations are generated using ParaView 
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9.2 Vector Field Magnitude and Angle 


As can be seen in Fig. 17, the velocity magnitude and structure of the flow directions is 
very well maintained down to fairly course reconstructions using R = 32. Especially 
the (important) regions with larger velocity magnitudes are very well preserved with 
respect to their flow orientation as visible from the streamline visualization. The 
rendering applies an opacity transfer function setting which is almost linear, slight 
curve below the diagonal, to the vector field magnitude, highlighting the important 
high-velocity regions. 

To see how many details are lost after rank reduction of the velocity field data 
set, we calculated and visualized the error information for both magnitude and angle 
deviation in Figs. 18 and 19, respectively. For the magnitude, we normalized the error 
to be the percentage relative to the local vector field magnitude value. The rendering 
uses an opacity mapping which is almost linear, slight curve below the diagonal, 
to the vector field magnitude. This makes low magnitude and low opacity areas, 
and their errors, transparent or dark and highlights any errors in the more critical 
high-velocity regions. The completely dark images in Fig. 18e—h demonstrate that 
the magnitude error after rank reduction using Tucker decomposition is close to zero 
even with for R = 16, which is 1/16 of the original full rank of 256. The barely 
noticeable dark blue regions in Fig. 18a—d correspond to low errors in high-velocity 
regions, with errors in the range of less than 10% down to 1078% (with 60% being 
white). 

Compared to the almost negligible relative magnitude errors of the velocity field, 
however, the angular error of the same data set after rank reduction is more prominent 
as shown in Fig. 19. The maximum angular error m = 180° is shown in red while 
small errors are shown in dark blue (0°). The rendering applies an opacity setting 
almost linear to the angle offset, hence in Fig. 19 large angular errors are highlighted. 


(a) R=32 (b) R= 64 (©) R= 128 (d) R =256 


Fig. 17 Scalar velocity magnitude (top) and vector field streamline (bottom) visualization of the 
forced MHD turbulence simulation at variable reduced rank reconstructions R = 32, 64, 128, 256 
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(a) R=1 (b) R=2 (c) R=4 (d) R=8 
(e) R=16 (f) R=32 (g) R=64 (h) R= 128 


Fig. 18 Relative magnitude error (in %) of the velocity field with reduced rank reconstructions 
R = 1, 2,4, 8, 16, 32, 64, 128 


CED 


(a) R=1 (b) R=2 (c) R=4 (d) R=8 
EF} | | j 
(c) R= 16 (f) R= 32 (g) R=64 (h) R = 128 


Fig. 19 Absolute angular error [0°, 180°] of the velocity field with reduced rank reconstructions 
R = 1, 2,4, 8, 16, 32, 64, 128 


However, this does not relate to the local strength of the vector field since large 
angle differences in low-magnitude areas are not that important. Directly comparing 
Figs 17 and 19, which have the same viewing configuration, one can observe that 
large angular errors occur mostly in very low-velocity areas and may thus not be 
that relevant. In particular, since for very short vectors small errors can cause large 
angular changes. 
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9.3 Vorticity and Divergence 


We also inspected the influence of low-rank tensor compression on the two most 
fundamental features of vector field data: divergence and vorticity. As can be seen 
in Fig. 20, the global structure of the magnitude from both the divergence field and 
the vorticity field are still very well preserved with rank R = 64, which is 1/4 of the 
original full rank of 256. The values are given in their absolute ranges of [0.36, —0.43] 
and [0.52, 1075] for the divergence and vorticity respectively. The rendering applies 
an opacity setting which is almost linear, slight curve below the diagonal, to the 
magnitude values, highlighting the important regions. 

For the vorticity vector field, we also calculated the relative magnitude error as 
for the velocity vector field data shown in Fig. 18, thus as percentage of the vorticity 
magnitude, and similar conclusions can also be drawn here for the results shown in 
Fig. 21. All errors are very low, and barely noticeable for rank-reductions down to 
R = 16, which corresponds to 1/16 of the original data volume. 

Additionally, we also computed the absolute angular error for the vorticity vector 
field, in this case applying an opacity proportional to the magnitude of the raw vector 
field, and the results are shown in Fig. 22. We can observe that the angular error 
almost everywhere is in the color range of dark to light blue which maps to errors 
from 0 up to 45° in this plot, and thus the directional vorticity information seems to 
be well preserved. 


(a) R=32 b) R= 64 (c) R= 128 (d) R =256 


Fig. 20 Magnitude of the vector field’s divergence (top) and vorticity (bottom) at variable reduced 
rank reconstructions R = 32, 64, 128, 256 
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(a) R=1 (b) R=2 (c) R=4 (d) R=8 
(e) R=16 (f) R=32 (g) R=64 (h) R = 128 


Fig. 21 Relative magnitude error (in %) of the vorticity field with reduced rank reconstructions 
R = 1, 2,4, 8, 16, 32, 64, 128 
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(a) R=1 (b) R=2 (c) R=4 (d) R=8 
(c) R= 16 (f) R= 32 (g) R=64 (h) R = 128 


Fig. 22 Absolute angular error [0°, 130°] of the vorticity field with reduced rank reconstructions 
R = 1, 2,4, 8, 16, 32, 64, 128 


10 Conclusions 


In the analysis conducted in this first study on vector fields, we have shown that tensor 
approximation methods are not only very useful for multidimensional scalar fields 
but can also be applied to multivariate data, thus extending to vector and possibly 
tensor fields. In general we can observe that for the important high-velocity vector 
field regions, the low-rank reconstructions maintain the important overall structures 
of the flow features, in particular also the vorticity. Furthermore, we note that the 
MHD simulation model in theory should be divergence-free and thus the numerical 
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results report very low divergence numbers. Therefore, we are very satisfied with 
the result that the low-rank tensor reconstructions do not result in an unexpected 
and uncontrolled enlargement of these divergence values, keeping them within the 
numerical range of the simulation. 
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Abstract Edge-enhancing diffusion (EED) can reconstruct a close approximation 
of an original image from a small subset of its pixels. This makes it an attractive foun- 
dation for PDE based image compression. In this work, we generalize second-order 
EED to a fourth-order counterpart. It involves a fourth-order diffusion tensor that is 
constructed from the regularized image gradient in a similar way as in traditional 
second-order EED, permitting diffusion along edges, while applying a non-linear 
diffusivity function across them. We show that our fourth-order diffusion tensor 
formalism provides a unifying framework for all previous anisotropic fourth-order 
diffusion based methods, and that it provides additional flexibility. We achieve an 
efficient implementation using a fast semi-iterative scheme. Experimental results on 
natural and medical images suggest that our novel fourth-order method produces 
more accurate reconstructions compared to the existing second-order EED. 


1 Introduction 


The increased availability and resolution of imaging technology, including digital 
cameras and medical imaging devices, along with advances in storage capacity and 
transfer bandwidths, have led to a proliferation of large image data. This makes image 
compression an important area of research. Image compression techniques can be 
divided into two main groups: Lossy and lossless compression. Lossless compres- 
sion techniques permit restoration of the full, unmodified image data, which however 
limits the achievable compression rates. Our work is concerned with lossy compres- 
sion, which achieves much higher compression rates by replacing the original image 
with an approximation that can be stored more efficiently. 
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We continue a line of research that has explored the use of Partial Differential 
Equations (PDEs) for lossy image compression [12, 13, 23, 28]. This approach is 
based on storing only a small subset of all pixels, and interpolating between them in 
order to restore the remaining ones. There is a strong similarity between that inter- 
polation process and image inpainting, whose goal it is to reconstruct missing or 
corrupted parts of an image. PDE-based methods for image inpainting and compres- 
sion are inspired by the physical phenomenon of heat transport. It is described by 
the heat diffusion equation 

ðu = div(D- Vu) , (d) 


which relates temporal changes in a heat concentration ðu to the divergence of 
its spatial gradient Vu. When diffusion takes place in an isotropic medium, the 
diffusivity D is a scalar that determines the rate of heat transfer. In an anisotropic 
medium, heat spreads out more rapidly in some directions than in others. In those 
cases, D is a diffusion tensor, i.e., a symmetric matrix that encodes this directional 
dependence. 

When applied to image processing, the gray value at a certain location is inter- 
preted as the heat concentration u. In diffusion-based image inpainting, Eq. (1) is 
used to propagate information from the known pixels, whose intensity is fixed, to the 
unknown pixels which will ultimately reach a steady state in which their intensity is 
determined by their surrounding known pixels. In this sense, Eq. (1) has a filling-in 
effect that can be exploited for image compression. 

Different choices of the diffusivity function D lead to different kinds of diffu- 
sion. Linear diffusion [20] and nonlinear diffusion [24] were widely used for image 
smoothing and image enhancement. Edge structures in images can be enhanced by 
employing a diffusion tensor which allows diffusion in the direction perpendicu- 
lar to the local gradient, while applying a nonlinear diffusivity function along the 
gradient direction. This idea has led to the development of anisotropic nonlinear 
edge-enhancing diffusion (EED) [34]. Among the six variants that were evaluated 
for image compression by Galić et al. [13], EED led to the most accurate reconstruc- 
tions. Subsequently, this idea was applied to three-dimensional data compression 
[25], and combined with motion compensation in order to obtain a framework for 
video compression [2]. When combined with a suitable scheme for selecting and 
storing the preserved pixels, a few additional optimizations, and at sufficiently high 
compression rates, anisotropic diffusion has been shown to beat the quality even of 
JPEG2000 [27]. 

In this paper, we introduce a novel fourth-order PDE that generalizes second-order 
EED, and achieves even more accurate reconstructions. We build on prior works that 
proposed fourth-order analogs of the diffusion equation, and used them for image 
processing [10, 19, 21, 22, 26, 35]. In particular, we extend a work by Gorgi Zadeh 
et al. [15], who introduced the idea of steering anisotropic fourth-order diffusion 
with a fourth-order diffusion tensor. However, their method focuses on the curvature 
enhancement property of nonlinear fourth-order diffusion [10] in order to better 
localize ridge and valley structures. Deriving a suitable PDE for image inpainting 
requires a different definition of the diffusion tensor, more similar to the one in edge- 
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enhancing diffusion [34]. Two anisotropic fourth-order PDEs for inpainting were 
previously introduced by Li et al. [21]. However, they only apply them to image 
restoration tasks in which small parts of an image are missing (such as in Fig. 8), 
not to the reconstruction from a small subset of pixels. Moreover, we demonstrate 
that the fourth-order diffusion tensor based framework is more general in the sense 
that it can be used to express anisotropic fourth-order diffusion as it was described 
by Hajiaboli [19] or by Li et al. [21], while providing additional flexibility. 


2 Background and Related Work 


We will now formalize the above-mentioned idea of diffusion-based inpainting 
(Sect. 2.1), and review two concepts that play a central role in our method: Anisotropic 
nonlinear diffusion (Sect. 2.2) and fourth-order diffusion (Sect. 2.3). Further details 
can be found in works by Galić et al. [13] and Weickert [34], respectively. Finally, 
we provide additional context with a brief discussion of alternative approaches to 
image compression (Sect. 2.4). 


2.1 Diffusion-Based Inpainting 


In order to apply Eq. (1) to image smoothing, we have to restrict it to the image 
domain Q, and specify the behavior along its boundary 32. It is common to assume 
that no heat is transferred through that boundary (homogeneous Neumann boundary 
condition). Moreover, the positive real line (0, 00) is typically taken as the time 
domain. The resulting PDE can be written as 


ðu = div(D- Vu), Q x (0,00), 5 

ðu =0, ƏQ x (0,0), 2) 
where n is the normal vector to the boundary 3 Q. The original image f : Q — Ris 
used to specify an initial condition u = f at t = 0. For increasing diffusion time t, 
u will correspond to an increasingly smoothed version of the image. 

In image inpainting, we know the pixel values on a subset K C Q of the image, 
and aim to reconstruct plausible values in the unknown regions. A diffusion-based 
model for inpainting can be derived from the one for smoothing, by modeling the set 
of locations at which the pixel values are known with Dirichlet boundary conditions. 
In this case, f : K — R will be used to model the known values. In inpainting-based 
image compression, K will consist of a small fraction of the pixels in the original 
image. With this, we obtain the following model for inpainting: 
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ðu = div(D - Vu), Q\K x (0,00) , 
0,4 = 0, ƏXQ x (0,00), (3) 
u = f, K x [0, 00) 


In this case, the diffusion process spreads out the information from the known 
pixels to their spatial neighborhood. For time t —> ov, image smoothing and inpaint- 
ing both converge to a steady state, i.e., lim;—oo 0;u = 0. However, the steady-state 
of smoothing is trivial (u approaches a constant image with average gray value), 
while the Dirichlet boundary conditions in the inpainting case ensure a non-trivial 
steady-state, which is taken as the final inpainting result: Uinpaintea = lim;— o 4. 


2.2 From Linear to Anisotropic Nonlinear Diffusion 


So far, we assumed that the diffusion coefficient D is a scalar and constant, indepen- 
dent from the location within the image. This results in an inpainting model based 
on linear homogeneous diffusion [20]. With D = 1, it can be written as 


ðu = Au, Q\K x (0,00). (4) 


In this and all remaining equations in this section, the same boundary conditions 
are assumed as specified in Eq. (3). Despite its simplicity, it has been demonstrated 
that using this inpainting model for image compression can already beat the JPEG 
standard when applied to cartoon-like images, and selecting the retained pixels to be 
close to image edges [23]. 

When the diffusion coefficient is a scalar but depends on u, i.e., D = g(u), then 
we call the model inpainting based on nonlinear isotropic diffusion [24]. A common 
variant is to make D depend on the local gradient magnitude, i.e., 


ðu = div(g(||Vug||")Vu), Q\K x 0,00), (5) 


where g is a decreasing nonnegative diffusivity function, e.g., the Charbonnier 
diffusivity 


1 
g(s’?) = ——_., (6) 


Ji+5 


and A is a contrast parameter separating low from high diffusion areas [8]. In order 
to localize edges better and to make the problem well-posed, the image is pre- 
smoothed with a Gaussian before taking its gradient, i.e., g (||V uo ||?) is used instead 
of g(||Vul|*) [6]. 
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In the above-discussed models, the diffusion occurs only in the gradient direction. 
This can be changed by replacing the scalar diffusivity with a second-order diffusion 
tensor, i.e., a symmetric positive definite matrix. This is the basis of anisotropic 
nonlinear diffusion [34], 


ðu = div(D- Vu), Q\K x (0,00). (7) 


In edge-enhancing diffusion (EED), the diffusion tensor D is defined as 


D = g(\|Vuoll”)- viv; + 1- vavy, (8) 
le 
where vı = mir and v2 = ie. This means that diffusion across the edge (v1) 


is decreased depending on the gradient magnitude, while diffusion along the edge 
(V2) is allowed. Examples of EED based inpainting are included in our experimental 
results. In general, EED based inpainting results in better interpolated images than 
linear homogeneous or nonlinear isotropic PDEs. This makes it a current state-of- 
the-art choice for PDE-based image compression. 


2.3 From Second to Fourth Order Diffusion 


All models discussed above, as well as several others that have been proposed for 
inpainting [36], share a common property: They rely on second order PDEs. In 
image denoising, higher-order PDEs have a long history, going back to work by 
Scherzer [26]. You and Kaveh [35] propose fourth-order PDEs as a solution to the 
so-called staircasing problem that arises in edge-enhancing second-order PDEs, such 
as the filter proposed by Perona and Malik [24]: While the second-order Perona- 
Malik equation creates visually unpleasant step edges from continuous variations 
of intensity, corresponding fourth-order methods move these discontinuities into the 
gradients, where they are less noticeable to the human eye [16]. Subsequently, other 
fourth-order PDE-based models have been introduced, and have mostly been applied 
for denoising [18, 19, 22]. 

For a specific family of higher-order diffusion filters, Didas et al. [10] have shown 
that, in addition to preserving average gray value, they also preserve higher moments 
of the initial image. Moreover, depending on the diffusivity function, they can lead to 
adaptive forward and backward diffusion, and therefore to the enhancement of image 
features such as curvature. Gorgi Zadeh et al. [15] made use of this property in order 
to enhance ridges and valleys, by steering fourth-order diffusion with a fourth-order 
diffusion tensor. Our work adapts their method in order to achieve accurate inpainting 
and reconstruction from a small subset of pixels. 
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2.4 Alternative Approaches to Image Compression 


The dominant lossy image compression techniques today are JPEG and JPEG2000. 
They are based on the discrete cosine transform (DCT) and wavelet transform, respec- 
tively. However, they are not sensitive to the geometry of an image, i.e. those stan- 
dards are not tailored to their geometrical behavior [5]. Especially, the JPEG standard 
involves dividing the image into small square blocks. This can cause a degradation 
called “blocking effect” [30], and can result in unsatisfactory reconstructions espe- 
cially at high compression rates. 

It is an ongoing research trend to apply machine learning methods to image com- 
pression, such as convolutional and recurrent neural networks [3, 31, 32]. Learning 
based approaches tend to perform very well on the specific class of images on which 
they were trained, but require a huge amount of data. For example, Toderici et al. 
[33] used for training a dataset of 6 million 1280 x 720 images taken from the web. 


3 Method 


We will now introduce our novel PDE (Sect. 3.1), investigate its relationship to 
previously proposed anisotropic fourth-order diffusion (Sect. 3.2), and comment on 
our chosen discretization, as well as numerical stability (Sect. 3.3). 


3.1 Anisotropic Edge-Enhancing Fourth Order PDE 


Our fourth-order PDE builds on a model that was proposed by Gorgi Zadeh et al. [15] 
for ridge and valley enhancement. It can be stated concisely using Einstein notation, 
where summation is implied for indices appearing twice in the same expression: 


du = —0;; [D(H (u)) : H(u)] (9) 


ij 
In this equation, H(u) denotes the Hessian matrix of image u. The “double dot 
product” T = 2D : H indicates that matrix T is obtained by applying a linear map D 
to H, and the square bracket notation [T];; indicates taking the (i, j)th component: 


[T]; = [Dp (uc)) : H(u)];; = [Dp Wo )) |. y [H (ut) Je (10) 
Since D maps matrices to matrices, it is a fourth-order tensor. Since its role is 


analogous to the second-order diffusion tensor in Eq. (7), it is referred to as a fourth- 
order diffusion tensor. 
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The diffusion tensor D in Eq. (9) is a function of the local normalized Hessian 
H, (u.) which contains the information that is relevant to achieve curvature enhance- 
ment. For image inpainting, we propose to instead steer the fourth-order diffusion 
in analogy to edge-enhancing diffusion, i.e., as a function of the structure tensor 
J(us), which is obtained from image u after Gaussian pre-smoothing with band- 
width ø. We construct our fourth order diffusion tensor D from its eigenvalues u; 
and eigentensors E; via the spectral decomposition: 


D(JUs)) = HiE, 8 E; + mE: 8 Ey + uE; 8 E; + u4E,4 8 E4 (11) 
The eigenvalues and eigentensors are defined as 


Hı = g(à1), E; =V 8V ’ 
2 =1, Fn =v. @vW2, 


1 
u3 = Vg(A1), E3 = fe eee (12) 


1 
H4 = 0, Ba ee =O 


where g is a nonnegative decreasing diffusivity function, A; and v; are eigenvalues 


and eigenvectors of the structure tensor J(u.) = VuoVul, i.e., A; = ||Vug| l2, v= 


VuŁ : 
K. ak and à> = 0, v2 = ive The above-defined eigentensors are orthonormal 


with respect to the dot product A : B = trace(B™A) [15]. 
Combining this new definition of the fourth-order diffusion tensor with Dirichlet 
boundary conditions as in Eq. (3) results in our proposed model: 


dbu = — lD JUs )) :HW)]ij, 2\K x (0, 00), (13) 
u = f, K x [0, œœ) 

As it is customary in PDE-based inpainting, we allow Eq. (13) to evolve until 
a steady state has been reached, i.e., the time derivative becomes negligible. In our 
numerical implementation, we use a Fast Semi-Iterative Scheme (FSI) [17] to greatly 
accelerate convergence to a large stopping time. 

In the definition of our fourth-order diffusion tensor D, the choice of u, and u2 
is analogous to anisotropic edge enhancing diffusion [34]. However, two additional 
terms occur in the fourth-order case, u3 and u4. As noted in [15], u4 is irrelevant, 
since the corresponding eigentensor E; is anti-symmetric, and the dot product E4 : H 
with the Hessian of any sufficiently smooth image will be zero due to its symmetry. 
To better understand the role of u3, we observe that 
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1 
E;:H= a (vi Hv2 + v) Hv) 


- lemon eD) 


which amounts to a mixed second derivative of u, in directions along and orthogonal 
to the regularized image gradient wi ae or, equivalently, to the difference of second 
derivatives in the directions that are exactly in between the two. This term vanishes if 
the Hessian is isotropic, or if the gradient is parallel to one of the Hessian eigenvectors. 
Therefore, the role of u3 can be seen as steering the amount of diffusion in cases 
of a Hessian anisotropy that goes along with a misalignment between gradient and 
Hessian eigenvectors. 

Gorgi Zadeh et al. [15] simply set u3 to the arithmetic mean of jz; and u2. In our 
work, we empirically evaluated several alternative options for u3 by reconstructing 
the test image shown in Fig. la, which contains one rectangle, one circle, and two 
stars, from a randomly selected subset of 5% of its pixels. In this experiment, we 
compare EED based inpainting with our novel fourth-order edge enhancing diffusion 
(FOEED) with different settings of u3: Specifically, y3 = 1 corresponds to the max- 


(14) 


 @ 
x x x * 


(a) Original test image of size (b) Randomly chosen 5% of (c) Second-order EED inpaint- 
300 x 300 pixel values ing based on (b) 


O © © 
xe k kk o 


(d) Fourth-order EED inpaint- (e) Fourth-order EED inpaint- (f) Fourth-order EED inpaint- 
— H+H? 


ing with u3 = 1 ing with u3 = 5 ing with u3 = quiu? 


Fig. 1 Reconstruction of a synthetic test image (a) from 5% of its pixels (b) based on second-order 
diffusion (c) and fourth-order diffusion with different coefficients for the mixed term u3 (d-f). 
Visually, the reconstruction in (f) is most similar to the original image 
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Table 1 Numerical reconstruction errors on the test image (Fig. 1) 
EED FOEED (u3 = 1) | FOEED 


FOEED 
(u3 = yA 2) 
533.987 

4.140 


Errors 


MSE 
AAE 


660.588 
4.581 


634.321 
4.505 


imum of u, and u (Fig. 1d), u3 = (1 + g(Aj)) /2 corresponds to their arithmetic 
mean (Fig. le), and u3 = \/g(A1) corresponds to their geometric mean (Fig. 1f). In 
all cases, we used the Charbonnier diffusivity (Eq. (6)), which is popular for image 
compression [13], the same contrast parameter (A = 0.1) and smoothing parameter 
(o = 1). The only difference is time step size, where second-order EED permitted a 
stable step size of 0.25, while a smaller step size of 0.05 was chosen for FOEED. A 
more detailed theoretical and empirical analysis of stability will be given in Sect. 3.3. 

A numerical comparison of the results is given in Table 1. For evaluation, we 
used the well-known mean squared error (MSE) and average absolute error (AAE) 
between original and reconstructed images. For two-dimensional gray-valued images 
u and v with the same dimensions m x n, the MSE and AAE are defined as 


1 
MSE(u, =— >% TED 
(u, v) an aM viedo) 
; J (15) 
AAE = — Juj- vijl. 
(u, v) Mn 3 |ti, j Vij 


According to Table 1, the most accurate results are achieved by setting m3 to 
the geometric mean of mı and u2. Fourth-order EED with this setting produces 
higher accuracy than second-order EED. Visually, Fig. 1 supports this conclusion. 
Specifically, fourth-order EED with u3 = .//41/42 is the only variant that correctly 
connects the thin bar at the top of the test image, and it leads to a straighter shape of 
the thicker bar below, which is more similar to its original rectangular shape. In all 
subsequent experiments, we set y3 = ,/{1[2. 


3.2 A Unifying Framework for Fourth-Order Diffusion 


Several fourth-order diffusion PDEs have been used for image processing previously. 
We can better understand how they relate to our newly proposed PDE by observing 
that the fourth-order diffusion tensor D introduces a unifying framework for fourth- 
order diffusion filters. In particular, given its coefficients dj ;,;, we can expand Eq. (9) 
by using Einstein notation as 


ðu = —ðjildijk uki] (16) 
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Effectively, the fourth-order diffusion tensor allows us to separately set the diffu- 
sivities for all 24 = 16 fourth-order derivatives of the two-dimensional image u. We 
will now demonstrate how several well-known fourth-order PDEs can be expressed 
in this framework, starting with the You-Kaveh PDE [35] 


du = —A(g(|Au|)Au) , (17) 


which can be rewritten as 


du = —Ixx[g([Aul|)uxy + 0+ xy + 0+ uyx + g(lAul)uyy] 
OO sit +O- xy +O: tyr + 0+ uyy] 

—ðky[O - uxx +O- uxy +O: uyr +0- uyy] 
—ðyylg(|Au]|)uxx +0 - uxy +0 - Uy, + g((Au])uyy] . 


(18) 


Here and in all subsequent examples, many terms have zero coefficients. For 
brevity, we will omit them from now on. 
Hajiaboli’s anisotropic fourth-order PDE [19] is 


du = —A (g(llVul) unn + g(llYulDurr) , (13) 


where N and T are unit vectors parallel and orthogonal to the gradient, respectively. 
It can be rewritten as 


: a [807D + 80VulDu5 _ (8&0Vul)?uxuy — g(l|Vuluxuy 
tu = XX 5) 2 Uxx 1 2 2 xy 
ux + Uy ux + Uy 
g0 Vul) uxuy — g(\Vulpuxty g(IVull) us + g(llVul u7 
Dy? Uyy + 24 ye yy 
ux + Uy ux + Uy (20) 
gl Vul)us + g(Vuldus) f 80lVul) uruy — g(lVuluxuy 
dyy 5) 7 Uxx T 72 7 Uxy 
ux + uy ux + uy 
g(lIVull)2uxuy — g(\[Vul)uxwy 8(llVul) u5 + g(\Vull uy 
i 24 ye Myx 4 ee) “yy 
ux + Uy ux + Uy 


From this method, Li et al. [21] derived two anisotropic fourth-order PDEs that, to 
our knowledge, are the only anisotropic fourth-order models that have been applied 
to inpainting previously. We will refer to them as Li 1 


ðu = —A(g(||Vul|)unn + urr) (21) 


and Li 2 
ðu = —A (urr). (22) 
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Li 1 can be re-written as 


V 24u? — uxi, 
TR (E u|| ur. “r)a + (Ss Vy 


2 2 2 2 
uy + Uy uz + Uy 


B(I|Vul|uxuy — Uyy g(\|Vul|)us + u? 
+ 7 7 Uyx +t | 375 Juw 
ux + Uy ux + Uy 


(23) 
g (EDE FHS) (g(llWul utety = uuy), 
yy u2 +u? XX u2 +u? xy 
8(I[Vul|uxuy — Uyty g(\|Vul|uy + u? 
F E 2 Uyx + lage 2 Uyy |, 
x y x y 


while Li 2 becomes 
2 2 
u —UxUuy —Uxuy u 
ðru = -ia| (ys + (Ss uw F (es Ju + (z) 
uy + Uy uy + uy ux + Uy ux + Uy 
2 2 
u —UxUy —UxUuy u 
-d (z) + ( 5 Žž z Jot (SS Jae + (athe any]: 
uy + Uy ux + Uy uy + Uy uy + Uy 


We observe that Li 1 is based on a similar idea as our proposed PDE: It permits 
fourth-order diffusion along the edge, while applying a nonlinear diffusivity function 
across the edge. However, expressing Li et al.’s models in terms of fourth-order 
diffusion tensors Dı and D, reveals that our approach is more general. In particular, 
we can observe that 


Dı : H= g(||Vul|unyl+ urri, 25) 
Do :H= urri 5 
where I is the 2 x 2 identity matrix. In our model, D : H can yield arbitrary 
anisotropic tensors. In this sense, our model more fully accounts for anisotropy 
compared to the ones by Hajiaboli and Li et al. 

The fourth-order Eq. (16) involves inner second derivatives of the image, which 
then get scaled by diffusivities, before outer second derivatives are taken. We observe 
that, in both cases, our model accounts for mixed derivatives that are ignored by 
previous approaches to anisotropic fourth-order diffusion: In the outer derivatives, 
this can be seen from the fact that Eq. (9) involves mixed derivatives, while Hajiaboli 
and Li et al. only consider the Laplacian. 

Similarly, our definition of a fourth-order diffusion tensor D accounts for mixed 
derivatives also in the inner derivatives. Following Eq. (14), we obtain 


D : H = u (Œ; 8 E1) : H + p2(K, 8 E2) : H+ u3 (E; @ E3) : H 


u 
= Hity, Ey + Hoty, E2 + 7 E = e) Ez. 
(26) 
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Comparing Eqs. (25) and (26) reveals differences in the considered directions: 
First, N and T are derived from the unregularized gradient, while the corresponding 
directions v; and v? in our model include a Gaussian pre-smoothing. A second 
difference is that our model involves an additional term, which is steered by u3, and 
accounts for the directions in between the regularized gradient and its orthogonal 
vectors, i.e., (=) and (=>). As it was demonstrated in the previous section, 
this term can have a noticeable effect on the outcome. Overall, we conclude that our 
newly proposed model is more general than the previously published ones. 


3.3 Discretization and Stability 


When discretizing Eq. (13) with standard finite differences 


(Uii, j — 2Ui j + uipi, j) 


Uxx ~ (Ax)? , 
(Ui j-i — Ui j + Ui jy) 
Uyy ~ 7 >, 
(Ay) (27) 
ae (uj—1,j—1 + tii, j1 + Ui-1,j41 + Ui4n,j—-1) 
a 4(Ax)(Ay) : 
lyy = lzy s 


we can write it down in matrix-vector form as in [15], 


ut! = u (I-t PP), (28) 
where u* is an mn dimensional image vector at iteration k. m,n are image width 
and height respectively; Ax and Ay are the corresponding pixel edge lengths. P 
is a positive semi-definite matrix that, with step size t, leads to the system matrix 
(I — t P,). The notation P; indicates that it is iteration dependent, i.e., Py = P(u,). 

Stability of fourth-order PDEs for image processing is typically formalized in an 
La sense, i.e., a time step is chosen such that 


k+1 k 
latl < |[u* lle - (29) 
In an inpainting scenario, it depends on our initialization of the unknown pixels 


whether we can expect Eq. (29) to hold. Therefore, we rely on a stability analysis of 
the smoothing variant of our proposed PDE. This variant is obtained by removing the 
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Dirichlet boundary conditions and instead solving a standard initial value problem. 
In this case, the stability analysis presented by Gorgi Zadeh et al. [15] carries over. 
It ensures that time step sizes 


2 
< 
T = 16(Ax)? + 16(Ay)? + 2(AxAy) 


(30) 


are stable in the L sense. For a spatial discretization Ax = Ay = 1, this yields 
t < 1/17 ~ 0.0588. In inpainting, we empirically obtained a useful steady state 
with a time step size t < 0.066, independent of the initialization. The fact that this 
slightly exceeds the theoretical step size reflects the fact that Eq. (30) results from 
deriving a sufficient, not a necessary condition for stability. 

Stability of fourth-order schemes generally requires a quite small time step t. This 
makes it computationally expensive to reach the steady state by evaluating Eq. (28). 
Hafner et al. [17] propose a remedy to this problem, the so-called Fast Semi-Iterative 
Scheme (FSI). It extrapolates the basic solver iteration with the previous iterate and 
serves as an accelerated explicit scheme. The acceleration of the explicit scheme (28) 
is given as 


weet =Q" a = tPU”) + a = a) . wk! , (31) 
where wu”! := wu” and a, = t for k = 0,...,n — 1. Here m stands for outer 


cycle, i.e. m-th cycle with inner cycle of length n. And for passing to the next 
outer cycle, we set u”*!-° := u™” , The stability analysis requires the matrix P to be 
symmetric. This is satisfied since the diffusion tensor D is symmetric, and symmetric 
central discretizations are used. In our implementation, we used n = 40, and stopped 
iterating after the first outer cycle for which |u” — u”~!||. < 1074. 


4 Experimental Results 


To establish the usefulness of our proposed new model, we applied it to the recon- 
struction of images from a sparse subset of pixels (Sect. 4.1). Moreover, we evaluate 
performance for a more classic inpainting task, scratch removal (Sect. 4.2). We 
also demonstrate how results depend on the chosen diffusivity function and contrast 
parameter (Sect. 4.3). 
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4.1 Reconstruction From a Sparse Set of Pixels 


Improving image reconstruction from a sparse set of known pixels was the main 
motivation behind our work. Therefore, we applied it to two well-known natural 
images, toucan and peppers, as well as to a medical image, a slice of a T; weighted 
brain MR scan (t/slice). For toucan, we kept a random subset of only 2% of the 
pixels. Due to the lower resolution of the peppers and t/slice images, we kept 5% 
and 20%, respectively. 

In all three cases, results from our approach (FOEED) were compared to results 
from second-order EED, as well as from the two anisotropic fourth-order PDEs 
proposed by Li et al. [21]. In all experiments, we used the Charbonnier diffusivity 
function, we set the contrast parameter to A = 0.1, and the pre-smoothing parameter 
too =1. 

Results for toucan are shown in Fig. 2, for peppers in Fig. 3, and for t/slice in Fig. 4. 
A quantitative evaluation in terms of MSE and AAE is presented in Table 2. In terms of 
the numerical results, our proposed method produced a more accurate reconstruction 
than any of the competing approaches. Visually, there is a clear difference between 
second-order (EED) and fourth-order approaches (Lil, Li2, FOEED). Especially, we 
found that the shapes of edges were reconstructed more accurately. For example, we 
noticed this around the body and face in the toucan image (Fig. 2). Similarly, the 
white and grey matter boundaries were better separated in the t/slice (Fig. 4). 

As we expected based on the theoretical analysis in Sect. 3.2, visual differences 
between the fourth-order methods are more subtle. However, in the peppers image 
(Fig. 3), the tall and thin and the small and thick peppers in the foreground are much 
more clearly separated in the FOEED result than in any of the others. 

In addition to experimenting with grayscale versions of the toucan and peppers 
images, we also applied EED and our FOEED filter channel-wise to the original 
RGB color versions. Results for toucan can be found in Fig. 5, for peppers in Fig. 6. 
Table 3 again provides a quantitative comparison. Similar observations can be made 
as in the grayscale images: Again, FOEED leads to lower reconstruction errors than 
EED, it visually reconstructs edges more accurately, and separates the peppers more 
clearly. 

Finally, we reconstructed images from a larger number of pixels, to obtain visually 
cleaner results. Qualitative and numerical results are presented in Fig. 7 and Table 4, 
respectively. FOEED still yields lower numerical errors than EED. Unsurprisingly, 
the differences become smaller and less visually prominent as the mask density 
increases. The table also reveals that FOEED requires more CPU time compared to 
standard EED. However, due to the use of FSI in both cases, the difference in running 
times until convergence is much lower than the difference in time step sizes. 
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Fig. 2 1st row left: original toucan image of size 512 x 512; right: randomly chosen 2% of pixel 
values; 2nd row left: EED based inpainted image; right: Lil based inpainted image; 3rd row left: 
Li2 based inpainted image; right: FOEED based inpainted image 
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Fig. 3 1st row left: original peppers image of size 225 x 225; Right: randomly chosen 5% of pixel 
values; 2nd row left: EED based inpainted image; Right: Lil based inpainted image; 3rd row left: 
Li2 based inpainted image; Right: FOEED based inpainted image 
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Fig. 4 1st row left: original t/slice image of size 256 x 256; Right: randomly chosen 20% of pixel 
values; 2nd row left: EED based inpainted image; Right: Lil based inpainted image; 3rd row left: 
Li2 based inpainted image; Right: FOEED based inpainted image 
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Table 2 Numerical comparison of inpainting models for gray-valued images 


Image Errors EED FOEED Lil Li2 
Toucan MSE 105.37 96.228 100.994 102.665 
AAE 4.488 4.164 4.397 4.465 
Peppers MSE 467.261 443.129 455.633 459.606 
AAE 10.94 10.523 11.042 11.107 
tl-slice MSE 166.356 150.002 152.698 155.955 
AAE 5.895 5.698 5.789 5.853 


Fig. 5 RGB toucan image, reconstructed from randomly chosen 2% of pixel values using EED 
(left) or FOEED (right) 


vw 


Fig. 6 RGB peppers image, reconstructed from randomly chosen 5% of pixel values using EED 
(left) or FOEED (right) 
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Table 3 Numerical comparison of inpainting models for RGB images 


Image Errors EED FOEED 
Toucan MSE 119.062 108.061 
AAE 4.819 4.594 
Peppers MSE 478.799 441.203 
AAE 11.049 10.543 


Table 4 Numerical comparison and computation times corresponding to Fig. 7 


Image Errors EED FOEED CPU time 
Toucan 18.029 17.295 53.060 (FOEED) 
1.696 1.686 21.259 (EED) 
Peppers 113.5 110.885 20.999 (FOEED) 
4.565 4.441 19.79 (EED) 
tl-slice 114.845 107.323 24.74 (FOEED) 


AAE 4.610 4.553 10.64 (EED) 


4.2 Scratch Removal 


Li et al. [21] proposed their anisotropic fourth-order PDE for more classical image 
inpainting tasks, such as scratch removal. We evaluated whether our more general 
filter can also provide a benefit in such a scenario by reconstructing a scratched 
version of the peppers image. Similar to Li et al., we first made the scratches rather 
thin, covering only 6% of all pixels. Results are shown in Fig. 8 and in Table 5. In 
this case, all methods work well: Numerical errors are small and similar between 
methods, and even though FOEED achieves the best numerical result, differences 
are difficult to discern visually. 

Therefore, we created a more challenging version with thicker scratches, covering 
18% of all pixels (Fig. 9). The corresponding numerical comparison is shown in 
Table 6. Here, FOEED achieves the most accurate reconstruction. Visually, we again 
observe that edges are reconstructed more accurately, and objects are more clearly 
separated, with fourth-order compared to second-order diffusion, and that steering 
it with a fourth-order diffusion tensor again provides small additional benefits over 
the previous methods. 


Table 5 Numerical comparison for peppers with thinner scratches (Fig. 8) 


Image Errors EED FOEED Lil Li2 
Peppers MSE 9.520 7.813 8.161 8.132 
AAE 0.363 0.326 0.346 0.346 
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Fig. 7 Higher quality reconstructions from a larger subset of pixels. 1st row: toucan image, recon- 
structed with EED (left) or FOEED (right) from randomly chosen 14% of pixels; 2nd row left: same 
for 20% of pixels from peppers; 3rd row left: same for 30% of pixels from t/slice. As expected, 
increasing the fraction of known pixels reduces the differences in the results of the two schemes 
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Fig. 8 Ist row left: original peppers image of size 225x225; Right: corrupted image. 2nd row 
left: EED based inpainting; Right: Lil based inpainting. 3rd row left: Li2 based inpainting; Right: 
FOEED based inpainting 
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Fig. 9 Ist row left: original peppers image of size 225 x 225; Right: corrupted image. 2nd row 
left: EED based inpainting; Right: Lil based inpainting. 3rd row left: Li2 based inpainting; Right: 
FOEED based inpainting 
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Table 6 Numerical comparison for peppers with thicker scratches (Fig. 9) 


Image Errors EED FOEED Lil Li2 
Peppers MSE 104.744 78.761 101.670 101.592 
AAE 2.455 2.099 2.465 2.450 


4.3 Effect of Diffusivity Function and Contrast Parameter 


For image inpainting with second-order PDEs, the Charbonnier diffusivity was pre- 
viously found to work better than other established diffusivity functions. To assess 
whether this is still true in the fourth-order case, we repeated the reconstruction of 
the peppers image as shown in Fig. 3 with different diffusivities. Table 7 summarizes 
the results. We conclude that the Charbonnier diffusivity still appears to be optimal. 

Finally, in Fig. 10, we illustrate how the reconstructed image depends on the 
contrast parameter à. As expected, increasing À leads to an increased blurring of 
edges. In the limit, the diffusivity function takes on values close to | over a substantial 
part of the image, and our model starts to approximate homogeneous fourth-order 
diffusion. 


Table 7 Numerical comparison of FOEED with different diffusivity functions 


Image | Errors Charbonnier [8] | Aubert [7] Perona- Perona- Geman- 
at. Gy? Malik [24] Malik2 [24] Reynolds [14] 
52 
VHD (2442/2 — acon 22 
1H) (s242)2 
Peppers | MSE 443.129 458.961 478.411 491.153 491.186 
AAE 10.523 10.587 10.943 11.157 11.007 


Ai 


Fig. 10 From left to right: FOEED based inpainted image with A = 0.1, à = 0.5, à = 15.5 
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5 Conclusions 


We introduced a novel fourth-order PDE for edge enhancing diffusion (FOEED), 
steered by a fourth-order diffusion tensor. We implemented it using a fast semi- 
iterative scheme, and demonstrated that it achieved improved accuracy in several 
inpainting tasks, including reconstructing images from a small fraction of pixels, or 
removing scratches. 

Our main motivation for using fourth-order diffusion in this context is the 
increased smoothness of results compared to second-order PDEs [35], which we 
expected to result in visually more pleasant reconstructions. The model in our current 
work is still based on a single edge direction at each pixel, extracted via a traditional 
second-order structure tensor. It is left as a separate research goal for future work 
to combine this with approaches for the estimation of complex structures such as 
crossings or bifurcations [1, 29], and with their improved reconstruction, e.g., by 
operating on the space of positions and orientations [4, 9, 11]. 

Finally, our current work only considered reconstructions from a random subset 
of pixels. A practical image compression codec that uses our novel PDE should 
investigate how it interacts with more sophisticated approaches for selecting and 
coding inpainting masks [27]. 
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Abstract Diffusion-Weighted Magnetic Resonance Imaging (DWI) enables the in- 
vivo visualization of fibrous tissues such as white matter in the brain. Diffusion- 
Tensor Imaging (DTI) specifically models the DWI diffusion measurements as a 
second order-tensor. The processing pipeline to visualize this data, from image acqui- 
sition to the final rendering, is rather complex. It involves a considerable amount of 
measurements, parameters and model assumptions, all of which generate uncertain- 
ties in the final result which typically are not shown to the analyst in the visualization. 
In recent years, there has been a considerable amount of work on the visualization 
of uncertainty in DWI, and specifically DTI. In this chapter, we primarily focus on 
DTI given its simplicity and applicability, however, several aspects presented are 
valid for DWI as a whole. We explore the various sources of uncertainties involved, 
approaches for modeling those uncertainties, and, finally, we survey different strate- 
gies to visually represent them. We also look at several related methods of uncertainty 
visualization that have been applied outside DTI and discuss how these techniques 
can be adopted to the DTI domain. We conclude our discussion with an overview of 
potential research directions. 
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1 Introduction 


Recent advancements in magnetic resonance imaging (MRI) technology have led 
to the development of various remarkable techniques for the interpretation of brain 
anatomy. The most promising one is diffusion-weighted imaging (DWI), the only 
non-invasive technique for the assessment of brain white matter connectivity. This 
approach relies on the measurement of anisotropic diffusion of water molecules. 
The imaging and the interpretation of the molecular diffusion have improved with 
the development of techniques like diffusion tensor imaging (DTI) and high angular 
resolution diffusion imaging (HARDI). In this chapter, we will discuss the visualiza- 
tion pipeline of DTI, given its clinical applicability. However, several visualization 
strategies and sources of uncertainties associated are valid for more advanced models 
like HARDI models. 

DTI allows direct in-vivo examination of the fibrous structure in the brain at a 
relatively low acquisition cost. By analyzing the three-dimensional shape of the diffu- 
sion tensor, it provides valuable information about the microstructure of brain tissues. 
Despite many advantages of this technique, some downsides limit its widespread use. 
The main reason is that the complexity in the data makes it notoriously difficult to 
infer and analyze. 

The DTI visualization pipeline consists of four main stages, from data acquisition 
to the final visual representation of the results, as shown in Fig. 1. Each stage is based 
on assumptions, parameters, and estimations subject to considerable uncertainties. 
The uncertainties involved in each of the pipelines’ stage can lead to unpredictable 
variations in the final output. 

Several state-of-the-art reports exist on DWI visualization [62, 92, 93, 99]. How- 
ever, none of them give an overview of uncertainty, or they focus on some specific 
aspects. Most of the visualization literature about uncertainty in DTI focuses on 
issues related to the visual representation rather than sources of error involved in 
the pipeline [36, 47, 92]. In this chapter, we discuss the DTI visualization pipeline 
and analyze the sources of uncertainties present at each stage. We briefly cover the 
approaches used for quantification of uncertainties, which are often omitted in other 
studies [36, 47]. We review state-of-the-art strategies for uncertainty visualization 
in DTI and compare their main characteristics and drawbacks. We further investi- 
gate several methodologies for uncertainty visualization in other domains that have 
not been explored in DWI and discuss how these techniques can be adopted in this 
domain. DWI models beyond DTI share a similar pipeline as the one shown in Fig. 1. 
However, some parameters, error sources, and visual representations would differ 
from the tensor model. Specifically, diffusion modeling and fiber tracking would be 
based on different parameters and algorithms. In this chapter, we will indicate the 
methods from the DTI that are valid for the more general DWI pipeline. 

In Sect. 2, we discuss the background and review the visualization techniques for 
DTI. In Sect. 3, we discuss the sources of uncertainties involved in the visualization 
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Fig. 1 The DTI visualization pipeline with sources of uncertainties at each step 


pipeline and proceed with the uncertainty modeling techniques in Sect. 4. We review 
uncertainty visualization methods in Sect. 5 and conclude with open issues and 
research direction in Sect. 6. 


2 Background 


Diffusion refers to the constant rapid movement of microscopic particles due to the 
presence of thermal energy, i.e., ‘Brownian motion’. DWI deals with the diffusion of 
water molecules present in biological tissues where the diffusion is usually restricted 
due to the hindrance by many obstacles such as axonal membranes, macromolecules, 
and myelin. This kind of restricted diffusion is known as anisotropic diffusion. Ste- 
jskal and Tanner [94] observed the anisotropic diffusion of water molecules in tissues 
and investigated the related modeling of the diffusion effects for MRI. The clinical 
application of this technique was first presented by Bihan et al. [61] with the intro- 
duction of diffusion MRI along with the concept of apparent diffusion coefficient 
(ADC). In some neurological conditions, the amount of diffusion is disturbed in the 
affected area. Through studying these changes in diffusion, the abnormalities can be 
detected. In the following section, we will summarize how these measurements have 
been used to visualize white matter tracts in the brain. 


2.1 Diffusion Tensor 


The pattern of diffusion anisotropy of white matter tracts in 3D space can be mathe- 
matically modeled by a second order tensor, called the diffusion tensor D, introduced 
by Basser et al. [5]. The tensor D is a symmetric, positive definite tensor represented 
by a3 x 3 matrix with six unique elements, denoted by Dj; as follows: 


Di; Dn Di3 
D = | Da Dy» Dy (d) 
D3ı D32 D33 
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The diagonal elements in the diffusion tensor D represent the diffusion coeffi- 
cients along the principle axes x, y and z, while the off-diagonal elements represent 
the correlation of the diffusion between each pair of the principal direction. The 
diffusion tensor D is symmetric about the diagonal axis (D;; = Dji). By analysis of 
the eigenvectors and eigenvalues A; ,A2,A3 of the diffusion matrix, the length and the 
direction of the principal axes of the diffusion tensor can be determined. 

The six unique values in the tensor D provide the intervoxel diffusion information 
and the microstructure of a particular voxel. However, the six-dimensional diffusion 
tensor is hard to infer and present to a user. For this reason, several scalar quantities 
have been introduced to simplify the tensor to a single value. Fractional Anisotropy 
(FA), the most widely used scalar measure in diffusion tensor imaging [9] , represents 
the extent of the diffusion anisotropicity. A lower value of FA indicates that the 
diffusion is free (FA = 0; isotropic), while a high value of FA implies that the 
diffusion is restricted to a single direction (FA = 1; anisotropic). Another popular 
scalar measure in DTI is mean diffusivity (MD), which represents the overall amount 
of diffusion. Many other scalar measures have been proposed based on the more 
complex behaviour of molecular diffusion and are explained in detail in surveys by 
Novikov et al. [75], Rajagopalan et al. [83] and Vilanova et al. [99]. 


Visualization strategies: A Glyph is a general term for geometrically plotted spec- 
ifier that represent multidimensional data values. Data information is mapped to 
glyph characteristics such as shape and color. Glyphs provide a way to represent the 
full six-dimensional information of a diffusion tensor by mapping the eigenvectors 
and eigenvalues to the orientation and shape of a geometric primitive. The most 
straight forward approach to visualize the diffusion tensor are ellipsoidal glyphs [82] 
as shown in Fig. 2a. The orientation of the ellipsoid represents the direction of the 
major eigenvector, while the length represents the corresponding eigenvalue. Westin 
et al. [104] introduced three metrics to measure linear (A; > A2,A3), planar (A,=A2 > 
A3) and spherical diffusion (A; = Az = 3). Figure. 2 represents the barycentric space 
of diffusion tensor shapes in which the three extremes (linear, planar, and spherical) 
are at the corner of triangles. Among several other proposed techniques, the super 
quadratic glyph is considered state-of-the-art for glyph-based tensor visualization. 
Instead of interpolating between ellipsoidal shapes, Kindlmann [57] represents the 
diffusion by superquadrics with shape parameters defined by the barycentric coor- 
dinates,as shown in Fig. 2b. 
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Fig. 2 Barycentric space of diffusion tensor shapes 
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Outside DTI, glyphs are also used to represent the orientation distribution func- 
tion (ODF) of the molecular diffusion, which is commonly estimated by models 
that go beyond the diffusion tensor. ODF specifies the overall diffusion in a given 
direction, integrated over displacement magnitudes [103]. Spherical polar plots [98] 
parametrized surfaces [77], the HARDI glyph [81] and the HOME glyph [89] are 
some of the common glyph based visualization techniques for representing ODFs. 


2.2 Fiber Tracking 


The diffusion tensor provides per-voxel information about the orientation of the 
underlying neural tracts by analyzing the derived eigenvectors. By combining this 
information with other scalar measure, e.g. FA, one can estimate trajectories of white 
matter bundles in 3D space. The process of virtual reconstruction of the neural fiber 
tract on the basis of the diffusion tensor field is named Fiber Tracking or Tractography 

[8, 71]. These techniques of generating brain anatomical connectivity from the 
diffusion information have been summarized in review articles [42, 72, 102]. Fiber 


a) Polylines ) Illuminated streamlines 
c) Cylindrical tubes ) Streamtubes 


Fig. 3 Visualization techniques for deterministic tractography. Images are generated using 
vIST/e [1] 
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tracking methods have found its way in many neurological applications [18, 31, 45, 
73, 74]. 

The fiber tracking strategies can be mainly divided into deterministic, probabilis- 
tic, and global geometric techniques. Deterministic techniques always produce the 
same output with the same set of inputs. Probabilistic techniques, however, add ran- 
domness in the tracking process to incorporate the inherent uncertainty. We defer an 
extensive discussion of probabilistic methods to Sect. 4. Global geometric methods 
deduce connectivity in the white matter by globally optimizing a certain cost function 
based on the diffusion tensor information and are outside of the scope of this chapter. 

Streamline tracing is the most commonly used algorithm for tractography. It is 
a deterministic technique that generates trajectories by integrating the vector field 
defined by the main eigenvector at each voxel position. The tracing process ends 
when the stopping criteria are met. Several constraints can be used as stopping 
criteria, such as maximum turning angle or FA, to limit tracts to the region where 
tensors realistically represent the fiber tracts. 


Visualization strategies: Line-based approaches are the most straightforward tech- 
nique to represent deterministic fiber tracts. Numerous strategies have been intro- 
duced for the visualization of the white matter tracts, such as thin polylines [70] 
illuminated streamlines [114] or cylindrical tubes [8]. Zhang et al. [112] introduced 
streamtubes to encode the local diffusion tensor information along the cross-section 
of the fiber tracts at each voxel. This technique has previously been used to represent 
the tensor field in fluid dynamics, where they were called Hyper-streamlines [25]. 
Figure 3 shows the most commonly used representations for deterministic tractog- 
raphy. 

So far, we have discussed the visualization pipeline methods used in DTI without 
the involvement of uncertainties. In the following sections, we will review the sources 
of uncertainty present in the pipeline, the modeling techniques and the strategies used 
to visualize them. 


3 Sources of Uncertainty 


Understanding the sources of uncertainties is essential to provide effective visualiza- 
tion. The DTI visualization pipeline involves complex stages of mathematical mod- 
eling, analysis, mapping, and rendering strategies, therefore, it is prone to uncertainty 
from various sources. Noise, patient movement, modeling residuals, and distortion 
from imaging artifacts produce uncertainty in the orientation of the diffusion tensor 
and are detrimental to fiber tracking algorithms. These uncertainties hamper the link 
between the data being measured and visualized. The sources of uncertainty involved 
at each stage of the DTI visualization pipeline are shown in Fig. 1. In this section, we 
will go through this pipeline and discuss the sources of error present at each stage. 
Even though we focus on the DTI modality, several of the sources of uncertainty are 
present in DWI pipelines that go beyond DTI. 
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3.1 Image Acquisition 


MRI-based techniques usually suffer from various acquisition errors such as noise, 
motion artifacts, partial volume effects, etc. Signal to noise ratio in DWI sequences 
is relatively high given that signal attenuation is being measured. The effect of noise 
on the fiber tracking output has been widely studied in literature [3, 46, 59]. There 
has been a growing trend of increasing the gradient direction in DTI acquisition 
to improve the tractography quality. However, this further increases the acquisition 
time. In HARDI, the gradient directions for acquisition are much higher than that of 
DTI and, therefore, it needs more time. With higher acquisition time, it is more likely 
that the subject move during the scan, which in turn, introduces misalignment in the 
acquired image. These kinds of artifacts are known as motion artifacts. Providen- 
tially, these misalignments can be corrected during the registration process. Several 
automated techniques have been introduced to remove this artifact [113]. The finite 
resolution of the results also affects the output of the process. The resolution of a 
clinical DWI acquisition is typically in the order of millimeter (mm) in each direc- 
tion, which is much lower than that of actual axons. Therefore, the signal values 
have to be averaged to be able to fit in a single image voxel. This loss of information 
is called the partial-volume effect (PVE). Several studies have been conducted in 
neurological literature to investigate the PVE in DTI [84, 85, 101]. Other sources 
of error during image acquisition involve magnetic distortion, scanner setting and 
others [14]. 


3.2 Diffusion Tensor Calculation 


In DTI, the diffusion of a water molecule is mathematically represented by a second- 
order tensor, known as the diffusion tensor. Numerous measurements are performed 
along various gradient directions to determine the molecular diffusion at each voxel. 
The least-squares method is the most commonly used fitting technique to calculate the 
diffusion tensor, but other more accurate regression procedures can also be used [5, 
6]. This fitting procedure introduces a fitting error and involves a model choice. 
Therefore it adds variation in the outcome of the DTI procedure. DTI technique can 
only estimate one dominant diffusion direction per voxel, and thus, is incapable of 
determining the structure where the multi-fiber direction is present and, therefore, 
results in unreliable outcomes. HARDI models emerge to overcome this limitation 
and able to model complex fibrous regions of the brain. It provides a way to estimate 
the multi-fiber populations that can then be used for robust tractography. HARDI 
models are more complex and usually introduce more parameters and choices to be 
determined than DTI. 


132 F. Siddiqui et al. 


3.3 Fiber Tracking 


Fiber tracking involves the reconstruction of the fibrous structure of the brain white 
matter by gradually following the local fiber orientation estimated from the diffusion 
tensor. There are several parameters in fiber tracking to control the tracking process, 
however, these parameters add variability to the fiber tracking results. There are four 
major sources of uncertainties in the fiber tracking algorithm: 


1. Region definition and filtering 
2. Numerical approximation 

3. Interpolation 

4. Stopping criteria 


Region definition and filtering: Regions are usually defined by the user to start, end, 
or control the fiber pathways. The seeding region refers to the starting point of the 
tracking process and defines the initial conditions for numerical integration. Regions 
are also used to extract a specific bundle of interest and filter out others to avoid 
visual clutter. The region definition in the fiber tracking process also adds variation 
in the outcome. Usually, these regions are defined manually, and therefore introduce 
an implicit user bias. A minor variation in the definition can result in largely different 
pathways. Recently, several techniques have been proposed to minimize the effect 
of seed region in the fiber tracking algorithms [21, 46, 102]. 


Numerical approximation: Different types of numerical approximation schemes 
can be implemented in the fiber tracking algorithm. Euler integration is the most 
straight forward technique [71] but it is a strong approximation. Higher-order meth- 
ods, such as 2nd or 4th order Runge-Kutta methods [8], are typically less sensitive to 
noise and can be used for accurate fiber tracking. The integration step, or step size, 
further affect the quality of these integration schemes [97]. 


Interpolation: During the numerical approximation process, most of the time, the 
sample position after each integration step lies between volume grid points, hence, 
interpolation is needed to estimate values, based on the neighboring grid points. 
Several studies have been conducted to address the effect of interpolation in fiber 
tracking [32, 109]. Various kinds of interpolation schemes are present, each result 
in different pathways, and therefore, add variability in the results. 


Stopping criteria: Different scalar measures, such as FA, MD, or curve angle, can 
be used as stopping criteria in the fiber tracking process. Fiber tracking algorithms 
are often highly sensitive to these values, meaning that a very small variation in 
the stopping criteria can lead to a very large change in the resulting fiber [97]. 
Brecheisen et al. [16] propose a visual exploration tool that allows users to investigate 
the behavior and sensitivity of DTI fiber tracking to stopping criteria. 

In fiber tracking algorithms for HARDI models the principal directions are 
extracted from a multifiber representation which adds another layer of complexity 
to the algorithms. 
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3.4 Visualization 


The visualization stage involves the mapping of the data into a geometric representa- 
tion or visual primitives that are finally rendered on to the screen. This process can be 
another source the uncertainty. Various photo-realistic rendering techniques are used 
to simulate real world lighting as exact as possible, but this further complexity adds 
uncertainty in the outcome. Lighting models and shadows enhance the structural 
perception of the fibers and as such improve the recognition of the spatial relations 
between tracts; however, the controlling parameters can add further variability in the 
final results. 


4 Uncertainty Modeling 


As discussed in the previous section, many sources of uncertainties are present at 
each stage of the DTI visualization pipeline that affect the outcome of the process. 
These uncertainties propagate through the pipeline adding uncertainty in the derived 
quantities including diffusion tensor and fiber orientations. Estimating the error dis- 
tribution of different sources is not a straight forward task. Different approaches 
have been used to model the uncertainty, however, each with pros and cons. We have 
classified the methods used for the uncertainty quantification into two categories: 


1. Analytical methods 
2. Stochastic methods 


4.1 Analytical Methods 


Analytical methods refer to approaches that provide an explicit mathematical formu- 
lation of the error distribution. These modeling techniques are based on the Bayes the- 
orem [56] and were first introduced by Behrens et al. [12] in DWI. They estimated the 
probability distribution function (PDF) of the fiber orientation by a Bayesian model. 
The main disadvantage of this modeling technique is that they rely on the assumption 
of prior and noise present in the data. These techniques are computationally inex- 
pensive, however, their dependence on the prior assumption limit their widespread 
use. Most of the Bayesian model-based techniques are often combined with random 
sampling methods, such as Markov Chain Monte Carlo (MCMC), to determine the 
distribution of model parameters [11, 12, 33]. The application of Bayesian model 
based methods in DTI and HARDI has been reported several times [48, 54, 64]. 
Shortest path algorithms are another useful approach for quantifying structural 
brain connectivity and were first introduced by O’ Donnell et al. [78]. This approach 
relies on computing the connections between regions of interest rather than connec- 
tions from a seed. Schober et al. [88] presented the distribution of the shortest path 
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as a Gaussian process over the solution to an ordinary differential equation (ODE). 
This strategy offers novel ways to quantify and visualize uncertainty arising from 
the numerical computation and allow marginalization over a space of feasible solu- 
tions. Hauberg et al. [40] extended this work and incorporated data uncertainty in 
DTI by sub-sampling the diffusion gradients and solving the noisy ODE. Several 
other studies using the shortest path algorithms in fiber tracking can be found in the 
literature [39, 63]. 


4.2 Stochastic Methods 


Describing the probability distribution analytically and propagating it through the 
visualization pipeline is extremely difficult and often not feasible. The alternative 
and the most straight forward way to estimate the probability distribution func- 
tion is to repeat the acquisition multiple times, this approach is called the bootstrap 
method [27]. However, for robust estimation of the PDFs, hundreds of data sets are 
required, which is not practical in a clinical setting. Several stochastic techniques 
were proposed to overcome this limitation [20]. Among them, the most widely used 
techniques are residual bootstrapping and wild bootstrapping [106]. These techniques 
rely on a single scan and estimate the probability distribution from the residuals that 
remain after fitting diffusion tensor to the data. In residual bootstrapping [24], the 
distribution is estimated by randomly assigning the residuals among gradient direc- 
tions. Another possibility is to resample the data based on randomly flipping the 
sign of the residuals by assuming symmetry in the distribution. The latter approach 
is called wild bootstrapping [23]. A detailed comparison of bootstrap methods has 
been presented by Chung et al. [20]. Stochastic bootstrapping has been widely used 
for DTI [50, 60, 79, 100]. These techniques generate multiple DTI volumes through 
stochastic simulations for estimating the probability distribution, however, they are 
computationally very expensive. 

Various stochastic algorithms were introduced to incorporate uncertainty in trac- 
tography by adding randomness in the tracking process. These techniques are called 
probabilistic tractography [11, 87, 95]. These algorithms estimate the probability 
density function of the fiber orientation at each voxel and determine the propagation 
direction by drawing random samples from the distribution. Probabilistic fiber track- 
ing is preferable in most cases as it takes uncertainty into account and can estimate 
the confidence interval for each reconstructed pathways, however, they are compu- 
tationally expensive [12, 26]. Koch et al. [58] propose to use Monte Carlo random 
walks for the estimation of the fiber connectivity. The fiber tracking algorithm pro- 
ceeds through each randomly selected neighboring voxel depending on the angle 
between the voxel’s main eigenvector and its connecting angle with the neighboring 
voxels. A similar approach has been used in other studies to establish a connectivity 
map in a probabilistic sense [10, 13, 34, 80]. Monte Carlo methods have also been 
used to generate fiber tracks based on random particle movement [38]. The PDF 
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obtained from the analytical methods can be used to perform tractography with these 
stochastic techniques [33, 48]. These studies are based on DTI, however, the concept 
is extendable to HARDI as well, but they are not used much in this context [64]. 


5 Uncertainty Visualization 


So far, we have discussed the sources of uncertainty present in the visualization 
pipeline and the methods used for their quantification. Visualization provides a way to 
communicate data effectively and efficiently, however, uncertainty is often omitted in 
the process. Visualizing uncertainty information in DWI can help assess the accuracy 
of the acquisition and modeling, which ultimately guide the users in making critical 
decisions. However, the visualization of complex data in itself is not straightforward, 
adding uncertainty representation to it further complicates the process. Issues of 
visual cluttering and loss of anatomical context are some of the few complications 
when visualizing uncertainties. 

In this section, we will survey the strategies used for the visualization of the 
uncertainties in DTI and also discuss some related techniques used in the HARDI 
model. We also summarize these strategies in Table 1. The modeling column refers 
to the uncertainty quantification techniques, such as stochastic, bootstrapping, or 
analytical methods. Domain indicates the application area of the study and ensemble 
column categorizes the method into the local or global level. The representation 
specifies the measure used for the aggregations of the ensemble, and finally, the 
visualization column indicates the technique used to display the uncertainties. The 
visualization of uncertainty in DTI can roughly be divided into two categories. 


1. Local uncertainty visualization 
2. Global uncertainty visualization 


5.1 Local Uncertainty Visualization 


Local representations of the uncertainty depict variation per voxel inside the vector 
or tensor fields. Glyphs are typically used to depict the voxel-wise information of the 
data. Several glyph-based techniques have been proposed to visualize the inherent 
local uncertainty in DTI. Jones et al. [50] proposed a method to represent the confi- 
dence interval of the main fiber direction by rendering an uncertainty cone, as shown 
in Fig. 4a. Basser et al. [4] used a similar technique to represent the main eigenvector 
and their associated uncertainties. This visualization approach allows the represen- 
tation of the main diffusion direction and the confidence interval concurrently, also 
described in Table 1. Schultz et al. [90] demonstrate a new glyph design, called 
HiFiVE, that provides a more detailed impression of the uncertainty. It represents 
the variation corresponding to the main eigenvector by rendering a double cone (blue 
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Table 1 Summary of uncertainty representation and visualization strategies 
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References Modeling Domain Ensemble Representation Visualization 

Jones et al. [50] | Stochastic DWI Local Direction Interval glyph 
interval 

Basser et al. [4] | Stochastic DWI Local Direction Interval glyph 
interval 

Schultz Stochastic DWI Local Probability HIfive glyph 

et al. [90] distribution 

Jones et al. [52] | Stochastic DWI Local Mean and Overlay glyph 
median 

Zhang - DWI Local Mean and Halo and texture 

et al. [110] variance 

Zhang - DWI Local Difference Overlay glyph 

et al. [111] encoding 

Tournier Stochastic DWI Local ODF mean and | Semi-transparent 

et al. [96] variance glyph 

Jiao et al. [49] Stochastic DWI Local ODF SIP Volume rendered 

glyph 

Basser et al. [7] | Analytical DWI Local Mean and Superimpose glyph 
covariance 

Abbasloo Analytical DWI Local Mean and Overlay/Animation 

et al. [2] covariance glyph 

Gerrits et al. [35] | Analytical Both Local Mean and Superimpose glyph 
covariance 

Wittenbrink Bootstrap Non-DWI Local Mean and Flow-field glyph 

et al. [108] variance 

Zuk et al. [115] | Bootstrap Non-DWI Local Probability Flow-field glyph 
distribution 

Hlawatsch Bootstrap Non-DWI Local Mean and Flow-field glyph 

et al. [43] variance 

Lodha et al. [65] | Bootstrap Non-DWI Local Interval Flow-field glyph 

Otten et al. [76] |- DWI Global Line and interval | Illustrative 

Hermosilla - DWI Global Line and interval | Illustrative 

et al. [41] 

Brecheisen Stochastic DWI Global Line and interval | Illustrative 

et al. [15] 

Corouge Bootstrap DWI Global Ensembles Spaghetti plot 

et al. [22] 

Bjornemo Stochastic DWI Global Ensembles Spaghetti plot 

et al. [13] 

Jones et al. [51] | Stochastic DWI Global Ensembles Spaghetti plot 

Hangmann Stochastic DWI Global Ensembles Color coded 

et al. [38] spaghetti plot 

Ehricke Stochastic DWI Global Ensembles Color coded 

et al. [28] spaghetti plot 

Enders et al. [29] | — DWI Global Fiber clusters Wrapped 

geometrical hull 
Chen et al. [19] |- DWI Global Fiber clusters Wrapped 


geometrical hull 


(continued) 
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Table 1 (continued) 


137 


References Modeling Domain Ensemble Representation Visualization 

Merhof - DWI Global Fiber clusters Wrapped 

et al. [68] geometrical hull 

Jones et al. [53] | Bootstrap DWI Global Ensemble/local Streamtubes 
estimates 

Wiens Stochastic DWI Global Ensemble/local | Streamtubes 

et al. [107] estimates 

Goldau Stochastic DWI Global Fiber density Stipples glyphs 

et al. [37] 

Hlawitschka Stochastic DWI Global Fiber density Stipples glyphs 

et al. [44] 

Goldau Stochastic DWI Global Fiber density Stipples glyphs 

et al. [36] 

Brown et al. [17] | Stochastic DWI Global Fiber density Confidence region 

Schultz Stochastic DWI Global Connectivity Confidence region 

et al. [91] Probability 

Kapri et al. [55] |- DWI Global Connectivty Volume rendering 
Probability 

McGraw Stochastic DWI Global Connectivity Volume rendering 

et al. [67] Probability 

Koch et al. [58] | Stochastic DWI Global Connectivity Density map 
Probability 

Parker et al. [80] | Stochastic DWI Global Connectivity Density map 
Probability 

Kaden et al. [54] | Analytical DWI Global Connectivity Density map 
Probability 

Schober Analytical DWI Global Ensembles Wobbly Spaghetti 

et al. [88] plot 

Hauberg Analytical DWI Global Ensembles Wobbly spaghetti 

et al. [40] plot 

Mirzargar Bootstrap Non-DWI Global Band Depth Wrapped 

et al. [69] geometrical hull 

Whitaker Bootstrap Non-DWI Global Band Depth Contour lines 

et al. [105] 

Ferstl et al. [30] | Bootstrap Non-DWI Global Line and interval | Wrapped 

geometrical hull 
Sanyal et al. [86] | Bootstrap Non-DWI Global Mean and std. Ribbon 


deviation 


color) and the density estimation of the uncertainty around it (represented as a gray 
surface), as shown in Fig. 4b. 

Another way to represent the uncertainty in multivariate data is to estimate its 
covariance. It does not only express the variance in each coefficient but also indi- 
cates their linear dependencies. Since the diffusion tensor is a second-order tensor, 
its covariance is represented by a fourth-order tensor, however, the visualization of 
the fourth-order tensor is rather difficult in this context. Basser et al. [7] presented a 
novel technique for the spectral decomposition of the fourth-order covariance tensor 
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Fig. 4 Glyphs with uncertainty encoding 


and introduced the concept of tensorial normal distribution. They proposed a glyph 
representation, called radial glyphs , which depicts the overall variance and a com- 
posite glyph for representing the eigentensor of the fourth-order covariance. They 
visualized the expected mean tensor and its standard deviation as three isosurfaces. 
Abbasloo et al. [2] highlight that the radial glyph does not convey the correlation 
with the mean tensor and also suffers from high visual complexity in the tensor field. 
They proposed a more intuitive approach for the visualization of the covariance by 
using multiple levels of detail. Unlike Basser et al., Abbasloo et al. visualize the con- 
fidence interval at each eigenmode separately by glyph overlays and used animation 
to visualize the differences in each mode. Gerrits et al. [35] pointed out the short- 
coming in both of these visualization techniques and proposed a generic approach 
that incorporates all the coefficients of the mean tensor and covariance in a single 
glyph. 

Various studies have been published concerning the representation of the tensor 
ensemble directly. Jones et al. [52] visualize the ensemble data simply by overlaying 
several glyphs. Although the superposition depicts the overall picture of the data, it 
adds visual clutter and occlusion during display. To remedy this, Zhang et al. [111] 
used transparency to minimize the occlusion. Abbasloo et al. [2] tried to minimize this 
problem by rendering the superimposed glyphs in complementary colors. Zhang et 
al. [110] proposed an approach to decompose the tensor data into three properties (i.e., 
scale, shape, and orientation), representing the structure of the underlying fibers, and 
measure the variation per property. A glyph based representation has been presented 
in this study to visualize the ensemble effectively. The variation in the ensemble is 
represented by Halo and texture over the surface as shown in Fig. 4c. 
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The orientation distribution function (ODF), associated with HARDI, specifies the 
overall amount of diffusion in a given direction. Unlike the diffusion tensor model, 
ODFs can have multiple maxima, and therefore are capable of modeling complex 
fibrous structure more accurately than DTI. However, this technique is computa- 
tionally expensive. The representation of the ODF itself is a challenging task and 
adding uncertainty information only increases the complexity. Jiao et al. [49] pro- 
posed a technique to visualize uncertainty over polar ODF glyphs by using a volume 
rendering technique. They introduced shape inclusion probability (SIP) function to 
represent the orientation uncertainty of the tensor. Tournier et al. [96] presented 
a method to visualize uncertainties associated with ODFs by using semitransparent 
glyphs. They represent the mean ODF by the opaque surface and the mean + standard 
deviation by the transparent surface, as shown in Fig. 4d. 

The visualization of uncertainty in a diffusion tensor is similar to the uncertainty 
representation in a vector field where orientation is considered important. Several 
glyph-based techniques exist in this scope. Wittenbrink et al. [108] presented a glyphs 
based representation of the uncertainty for atmospheric and oceanographic data. 
Likewise, Hlawatsch [43] and Lodha et al. [65] visualize the local uncertainty in a 
fluid flow field using glyphs. Zuk et al. [115] proposed a glyph design to provide 
uncertainty information in a bidirectional vector field. These techniques rely on the 
representation of the vector direction and magnitude with encoded uncertainties to 
depict the local uncertainty present in the field. 


5.2 Global Uncertainty Visualization 


In contrast to the local strategies, global uncertainty visualization in DTI aims at 
providing information on how accurate fiber tract information is throughout the com- 
plete tensor field, and how the inherent uncertainties accumulate during the tracking 
process. In DWI independently of DTI or HARDI models being used, probabilistic 
tractography is often used to incorporate these uncertainties. The most widely used 
approach to visualize fibers obtained through probabilistic tractography is to super- 
impose the resulting fibers in a so-called spaghetti plot [13, 22, 51], see Fig. 5a. 
This visualization technique, however, does not depict a clear view of the region- 
wise fiber connections and its uncertainty and suffers from strong cluttering. Color 
coding the fiber tracts according to their seed points [22] does not suffice to mini- 
mize the complexity of the visualization. Schober et al. [88] and Hauberg et al. [40] 
used wobbly spaghetti plot that emphasize the fact that the individual resulting paths 
cannot be considered as real fibers in the brain which is a common misinterpretation 
in spaghetti plot. Instead, they are uncertain estimates of fibers. 

To overcome the complexity and clutter caused by the multiple superimposed 
tracts, Enders et al. [29] presented a technique to group the fibers related to a cer- 
tain nerve tract and generate a surface that wraps the resulting fibers. Similarly, 
Mehrof et al. [68] and Chen et al. [19] presented a method to cluster the fiber with 
a proximity-based algorithm and generate hulls encompassing the fiber bundles, as 
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(b) Wrapped streamlines [19] 


(d) Connectivity mapping [55] 


(c) Illustrative visualization [15] 


Fig. 5 Global uncertainty visualization strategies 


shown in Fig. 5b. The anatomical grouping helps the user to understand the under- 
lying fibrous structure. Outside of DTI, Frest et al. [30] used a similar technique to 
visualize uncertainty in flow field ensembles. They performed principal component 
analysis to cluster the streamlines in a low dimensional space and determine the mean 
and confidence interval in an ensemble. These representations are visualized with a 
line enclosed by a transparent surface. The geometrical hulls and enclosed surfaces 
reduce clutter, however, they cannot resolve complex cluster shapes. To alleviate 
these problems, Illustrative techniques have been proposed to represent the confi- 
dence interval of the fiber bundle by creating silhouette, outline, and contours [15, 
76], as shown in Fig. 5c. 

To improve the understanding of ensembles of curves, it has been proposed to 
visualize the statistical information such as mean or confidence intervals rather than 
the direct ensemble visualization as spaghetti plot. Table 1 indicates the various 
representations used by the studies. These representations, e.g, mean and confidence 
interval, are the summarization of the raw samples. Unlike scalar values, the statistical 
measures are not well defined for curves, and therefore, several approaches have been 
proposed for the estimation of these terms. Brecheisen et al. [15] proposed to compute 
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median and confidence intervals based on pre-selected distance measures between 
curves. In the field of fluid dynamics, a band-depth concept [66] has been introduced 
to analyze curve ensembles in two-dimensions [105] and three-dimensions [69]. This 
concept provides a way to determine centrality within the present curves and estimate 
the variations. Sanyal et al. [86] visualize the uncertainty in the wind trajectories by 
creating a ribbon along the ensemble mean. The width of a ribbon represents the 
variability at each point. 

A widely used approach for the visualization of the global uncertainty is to repre- 
sent and visualize measures derived from the probabilistic tractography. Voxel-wise 
fiber density computes the probability that a fiber tract traverses a voxel for a given 
seed region [17]. Voxel-wise fiber density [28, 38] helps to infer the anatomical 
connections. Another measure is the connectivity probability, which represents the 
probability of a fiber tract crossing a given voxel while connecting two fixed anatom- 
ical regions [91]. Von Kapri et al. [55] and McGraw et al. [67] used volume rendering 
for the visualization of density maps, as shown in Fig. 5d. The global visualization 
of the fiber tracts does not provide the local tensor information. To visualize the local 
uncertainty along with the probabilistic tracts, a stream tube technique has been pro- 
posed [53, 107], which maps the local uncertainty measure onto the cross-section of 
the tube. 

A common problem with the three-dimensional approaches is that the geomet- 
rical representation often occludes the underlying information, hampering its inter- 
pretation. Various slice-based methods have been proposed for the visualization of 
probabilistic fibers [58, 80]. These techniques have been used in neuroscience as 
they provide a way to directly visualize the anatomical information, making it easy 
to interpret anatomical context. Goldou et al. [36, 37] presented a novel slice based 
approach for visualizing the probability by rendering fiber stipples. The number of 
stipples, present at a particular region depicts the fiber density. Hlawitschka et al. [44] 
proposes to use poisson-disk sampling for the generation of the fiber stipples. 

Table | summarizes the survey indicating the domain, representation, and visu- 
alization strategies used to display the uncertainty. The table covers the approaches 
used for local and global uncertainty visualization in both the DWI and non-DWI 
domain. 


6 Conclusion 


Diffusion-weighted imaging relies on complex stages of signal acquisition, math- 
ematical modeling, model assumptions, and hence, is exposed to many sources of 
uncertainty. Excluding this uncertainty from the visualization does not only affect 
the result but also cripples the user to make an effective decision. However, the 
efficient visualization of the uncertainty in DWI is nontrivial, as the data itself has 
high visual complexity and adding uncertainty to it only adds further complexity. In 
this chapter, we explored uncertainty in the various stages of the DTI visualization 
pipeline. Several of the problems and solutions discussed throughout this chapter are 
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also valid for other models beyond Diffusion tensor, such as HARDI models. Even 
though we have not covered the technical background, where applicable, we have 
discussed the applicability of the strategies beyond DTI. Further, we have reviewed 
applicable uncertainty visualization techniques beyond the DWI domain. 

DWIis still a growing field, considering the recent advancements and the frequent 
development of new techniques, this survey should not be considered complete, it 
rather should be enhanced in the future. Studies on uncertainty visualization so far are 
mostly focused on the research aspect, however, no uncertainty visualization solution 
exist to specifically support clinical tractography. Visual analytics, an emerging field 
in visualization, can be helpful in enabling detailed analysis of uncertainty present 
in DWI data, building on the top of the studies present in this survey. Most of the 
presented studies deal with uncertainty on the noise and modeling level, dealing with 
other sources of uncertainty and visualizing them as a whole part of the exploration 
is another open research direction. 

We focus on the visualization pipeline and techniques mostly related to DTI, a 
study including other modeling techniques would be beneficial. In summary, even 
though the uncertainty visualization in DWI has evolved considerably in the last few 
years, we believe, a lot of work still needs to be done for the effective visualization 
and exploration of DWI data. 
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Abstract Tractography aims at describing the most likely neural fiber paths in white 
matter. A general issue of current tractography methods is their large false-positive 
rate. An approach to deal with this problem is tractogram filtering in which anatom- 
ically implausible streamlines are discarded as a post-processing step after tractog- 
raphy. In this chapter, we review the main approaches and methods from literature 
that are relevant for the application of tractogram filtering. Moreover, we give a per- 
spective on the central challenges for the development of new methods, including 
modern machine learning techniques, in this field in the next few years. 
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1 Introduction 


Diffusion magnetic resonance imaging (dMRIJ) is currently the method of choice for 
assessing the local microstructure in the white matter (WM) of the human brain in 
vivo. Tractography methods use this local microstructure to generate streamlines aim- 
ing at modeling the underlying anatomy of neural fibers in the brain. These stream- 
lines are locally aligned with the estimated tissue orientation. The set of obtained 
streamlines is usually referred to as a tractogram and is the basis for different sub- 
sequent analyses. 
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Tractography has potential for clinically-relevant applications. For instance, visu- 
alizations of tractography data are used by neurosurgeons with the goal of preserving 
important neural bundles during brain tumor resection [9, 16, 64]. Moreover, it is 
possible to group streamlines into bundles and perform analyses based on these. 
This so-called bundle-based or tract-based analysis (BBA/TBA) allows to compute 
statistics for individual bundles, either averaged or along the streamlines. This can 
be used to perform group comparisons [8, 10, 60, 70] or to analyse WM changes 
in patients over time [5]. Focusing on the tractogram as a whole, the concept of 
structural connectivity is employed for the analysis of how pairs of cortical and sub- 
cortical regions are connected through streamlines of the tractogram [49, 59]. These 
connections can be summarized in graphs, which can be seen as estimations of the 
actual anatomical connections between GM regions through neural pathways in the 
WM. Such graphs serve as the basis for different group comparisons. For example, 
this approach has been applied to different neurological diseases, including among 
others epilepsy (e.g., [21]), multiple sclerosis (e.g., [31]) and Alzheimer’s disease 
(e.g., [17, 41]), as well as for assessing differences in the brain due to normal aging 
(e.g., [13]). 

Despite its potential in the aforementioned applications, the validation of trac- 
tography is an open challenge in the field. Several recent studies have unveiled that 
state-of-the-art methods to construct tractograms suffer both from false positives 
(FP), i.e., streamlines that are not related to anatomical structures [32], and false 
negatives (FN), i.e., missing streamlines to accurately describe known WM anatomy 
in its whole extent [2]. At the same time, findings indicate that tractography results 
are to some extent reproducible [34]. That means that FPs are an intrinsic problem 
of current methods [32]. Already in 2014, Thomas et al. hypothesised that there 
are inherent limitations of tractography that lead to effects of both, FPs and FNs 
[62]. The importance of these two problems is application dependent. For example, 
it has been argued that for structural connectivity analysis, a high specificity is of 
higher importance than sensitivity [71]. This means that, for this specific application, 
reducing FPs is more important than reducing FNs. 

Despite the efforts in recent years, new approaches have still not overcome the 
limitations of tractography methods [53]. In the particular case of FP, an alternative 
to improving tractography methods is to remove the FPs from the tractogram in a post 
processing step. In the following, we refer to this approach as tractogram filtering. 
Several methods from the literature can be used for this goal. Even though some 
methods were not designed for tractogram filtering, they can be adapted for this 
purpose. In this chapter, we consider tractogram filtering as a binary classification 
problem in which we aim at assigning either a positive (P) or a negative (N) label to 
each streamline. This allows us to assess the possibility to use a particular method to 
define the labels P and N for, ideally, separating true positive (TP) and FP streamlines 
in a tractogram. In the following, we review the most relevant methods in literature, 
point out their issues and pose key challenges on the way to improved tractogram 
filtering. 

The chapter is organized as follows. Section 2 describes the main approaches and 
methods for tractogram filtering. Section 3 describes the key challenges of tractogram 
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filtering and gives a perspective on future developments in the field. Finally, we make 
some concluding remarks in Sect. 4. In order to illustrate specific properties or issues 
of the different strategies, we run some experiments on a very limited amount of data 
from the Human Connectome Project (HCP). 


2 Approaches for Tractogram Filtering 


In the following sections, we review methods that can be used for tractogram filter- 
ing. Based on the definition of the labels P and N for each streamline, we identified 
four main criteria that these methods build upon, namely: explainability of the diffu- 
sion signal, inclusion and exclusion regions of interest (ROIs), streamline geometry 
or shape, and streamline similarity and clustering. Figure 1 shows the problem of 
tractogram filtering interpreted as a binary classification problem and lists the most 
relevant methods in this realm together with the main criteria used to group the 
streamlines. 

Table | lists the most relevant tractogram filtering methods that are reviewed in the 
next subsections. The columns of this table describe the most relevant characteristics 
of these methods, namely: criteria, use of dMRI data, required context, main target 
and whether or not the method is data driven. First, criteria refers to the strategies 
from Fig. 1 that are followed when employing the method for streamline classifica- 
tion. Column dMRI in the table indicates if the method used the dMRI data or not. As 
shown, only a few filtering methods make use of the acquired dMRI for performing 
the classification. The listed methods also differ in the required streamline context. 
While some are able to perform the classification individually per streamline, others 
also require the streamlines in the bundles of interest or the complete tractogram as 
extra inputs (column context). Furthermore, by design, the target of some methods 
is to define either the positive (i.e., P) or negative (i.e., N) label but not both. In 
other words, some methods aim at being more specific detecting TPs than FPs, and 
the other way round for others. As an example, a streamline classified as negative 
by Recobundles can still be a TP, since it can belong to a bundle not present in the 
atlas. The target of the method is important to be considered, since preferring higher 
specificity of P or N is application dependent. This is shown in column target in 
the table. Finally, the methods can also be grouped into rule-based and data-driven 
ones. While the former makes use of classical approaches, the latter use machine 
learning techniques for performing the classification. 

In the following sections, we describe the aforementioned criteria in detail and list 
the most representative methods that use those criteria to perform the classification 
of streamlines. 
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Fig. 1 Tractogram filtering can be seen as classifying each streamline in a tractogram as either 
positive (P) or negative (N), targeting true positives (TPs) and false positives (FPs), respectively. 
Relevant methods use one or more of the depicted criteria for performing this classification. Some 
representative methods are also depicted in the figure. For a detailed list refer to Table 1 


2.1 Explainability of the Diffusion Signal 


The idea behind this approach is that high-quality tractograms can be used to explain 
the acquired dMRI data. In other words, synthetic dMRI generated from tractograms 
should be very similar to the acquired dMRI data. Thus, these methods focus on find- 
ing a subset of streamlines which can generate data that approximates the measured 
signal as closely as possible. Streamlines not belonging to such a subset are likely 
implausible (or duplicates of other streamlines already contributing to the signal) 
and might be removed. This approach is shown in Fig. 2. 
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Table 1 Representative methods for tractogram filtering. Criteria shows the criteria used for 
streamline classification (cf. Fig. 1). Column dMRI indicates if the method uses diffusion data. 
Context describes the contextual information required for streamline classification. Target shows 
the target labels where the method is more specific. Data driven specifies if the method makes use 
of machine learning or not 


Method Criteria dMRI Context Target Data-driven 
LiFE [42] dMRI Xl Tractogram = 
explainability 
COMMIT [12] | dMRI K Tractogram — 
explanability 
SIFT [55] dMRI K Tractogram = 
explainability 
SIFT2 [57] dMRI K Tractogram = 
explainability 
TractQuerier ROIs - - 
[66] 
TractSeg [67] ROIs K - P K 
FiberNet [24] Geometry - P K 
FiberMap [73] | Geometry - P Z 
TRAFIC [37] Geometry - P Z 
DeepBundle Geometry - P Z 
[30] 
Geometric DL | Geometry - N K 
[1] 
Recobundles Streamline Bundles P 
[19] similarity 
Curated WMA | Streamline Bundles P 
[74] similarity 
FS2NET [40] Streamline Bundles P K 
similarity 
DeepFiltering | dMRI pattern K - - x 
[25] 
BundleMAP Streamline Bundles P x 
[27] similarity, ROIs 
ExTractor [43] | Geometry, Tractogram N 
ROIs, 
Streamline 
similarity 
COMMIT2 [50] | dMRI Xl Tractogram - 
explainability, 
ROIs 
AnchorTracts dMRI xX Tractogram P 
[36] explainability, 
ROIs, 
Streamline 
similarity 
FiberNet2.0 [23] | ROIs, Geometry - P K 
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Fig. 2 dMRI signal fitting approach. Synthetic dMRI is computed based on a microstructure model 
and compared to the acquired dMRI. The streamlines that are not relevant to minimizing the residual 
might be filtered out 


As shown in Table 1, these methods require the whole tractogram as an input. In 
the following subsections, we will describe the most commonly used methods for 
dMRI signal fitting and discuss their issues. 


2.1.1 LiFE and COMMIT 


Linear Fascicle Evaluation (LiFE) [7, 42] and Convex Optimization Modeling for 
Microstructure Informed Tractography (COMMIT) [12] state the problem as follows: 
let y be a vector with the acquired diffusion signals, A(T) be a forward operator for 
synthesizing diffusion data from the streamlines of the tractogram T, x be a vector 
with weights for the contribution of every streamline to the acquired data, and ņ be 
the acquisition noise. Then y can be written as: 


y = A(T)x +n. (1) 


Since the weights x cannot be negative, it is possible to solve (1) through non-negative 
least squares: 
arg min||A(T) x — yll (2) 
x> 


Filtering is performed by discarding streamlines with low weights. This formulation 
allows for the use of different models to couple the information from streamlines 
to the measured signal or derivatives thereof. First, A can be chosen from a large 
variety of forward operators proposed in the literature [39]. As an example, in the 
original papers, COMMIT was based on a multi-compartment forward model while 
in LiFE the stick and ball model was used. Notwithstanding, both can be adapted to 
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use any other model. Second, different solvers can be applied for solving the non- 
negative least-squares problem of (2). Both COMMIT and LiFE use the subspace 
Barzilei-Borwein (SSB) solver proposed in [28]. Due to the nature of the problem, 
sparsity on weights is desirable. For this purpose, COMMIT proposes a basis pursuit 
de-noise (BPDN) formulation of (2) that actively considers sparsity by minimizing 
the !-norm of x. Such a formulation can be written as: 


argmin||x||;, subject to ||A(T) x — y||3 < €, (3) 
x>0 


where € is a parameter. 

In order to reduce the inherent computational burden of these strategies, A is 
implemented in LiFE and COMMIT through a lookup table on a dictionary of pre- 
computed estimations. Moreover, a GPU-based optimized version has recently been 
proposed for LiFE [29]. 


2.1.2 SIFT and SIFT2 


Very similar to the techniques from the previous subsection is Spherical- 
deconvolution Informed Filtering of Tractograms (SIFT) [55]. Instead of targeting 
the raw dMRI data, it aims at reconstructing the fiber orientation distribution function 
(fODF) in each voxel. First, the fODF is obtained with constrained spherical decon- 
volution (CSD) [63]. Second, the contribution of every streamline to the fODF is 
assessed. These contributions are used to determine whether a streamline is deemed 
redundant/noisy or not. Third, these contributions are sorted in order to remove the 
least relevant streamlines. Finally, the aforementioned two steps are iterated until 
either a target number of streamlines or a certain residual level is reached. Unlike 
LiFE and COMMIT, SIFT does not generate weights per streamline. Thus, SIFT2 
was proposed as a slight modification of SIFT in which an additional regularization 
term is added and a weight per streamline is computed [56]. 

In order to compare the agreement between SIFT and SIFT2, we run them with 
their standard parameters on a whole-brain tractogram computed with anatomically- 
constrained tractography (ACT) [54] from MRrtix3! with one million streamlines 
obtained from one HCP subject. In this experiment, SIFT selected 34.6% of the 
streamlines in the original tractogram. Figure 3 shows the histograms of weights 
computed with SIFT2 where the individual histograms are obtained by separating 
the streamlines based on whether they were accepted or discarded by SIFT. It can be 
seen that the two histograms have a big overlapping region. A repetition of the exper- 
iment with 500k streamlines showed similar results. This means that it is difficult to 
reproduce the results from SIFT with the weights computed with SIFT2. Thus, while 
SIFT2 can be useful for describing the contributions of streamlines to the acquired 
data, unlike SIFT, its direct use for tractogram filtering is not straightforward. 


‘https://www.mrtrix.org/. 
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Fig. 3 Histograms showing the frequency of different SIFT2 weights computed for streamlines 
that are filtered out (in blue) or not (in orange) with SIFT. Both methods were computed on an HCP 
subject with | million streamlines computed with ACT in MRtrix3 


2.1.3 Issues of dMRI Signal Fitting 


While dMRI signal fitting is appealing, it has not been able to significantly reduce the 
false positive rate [50, 52]. This fact can be attributed to various reasons. Daducci 
et al. [11] discuss a number of issues of dMRI signal fitting. First, most of the 
current dMRI signal fitting methods require a whole brain tractogram, even if a single 
white matter bundle is of interest. This results both in an unnecessary computational 
burden and a higher risk for false positives when targeting specific fiber bundles. 
For example, if a fiber bundle is not appropriately represented in the full tractogram, 
which is not uncommon, there is a higher risk for implausible streamlines from other 
bundles to take their place in the reconstruction. Second, the computed weights of 
streamlines tend to be inversely proportional to the number of similar streamlines in 
the tractogram. This effect is shown in Fig. 4 for the case of SIFT2 (a similar behavior 
is expected from LiFE and COMMIT). As shown, the SIFT2 weights are lower in 
the centerlines of the bundles, where tractography tends to yield more streamlines. 
Thus, thresholding of SIFT2 weights cannot be used for progressive filtering, since 
that might result in discarding the most important tracts very early. Moreover, some 
noisy streamlines might be classified as acceptable just because of the reward they 
get for reaching distant regions. This issue comes from the fact that weights of SIFT2 
are designed for fitting the acquired data, but not for filtering. This could potentially 
be solved through an extra step of weight normalization that, to our knowledge, has 
not been proposed so far. Third, as discussed in [11], minimizing the residual from 
Fig. 2 does not guarantee that the solution is plausible as the current methods are very 
prone to overfit due to the large amount of unknowns that must be estimated. Finally, 
working with incomplete streamlines can lead to biased results in the uncovered 
regions, which might be especially problematic for structural connectivity. 
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Fig.4 Average weights computed through SIFT2 per voxel. Darker and brighter voxels correspond 
to lower and higher SIFT2 weights, respectively. The values are extracted from the same dataset of 
Fig. 3 


Fig. 5 The course of two close fiber bundles is depicted in dashed green. A local error (blue box) 
can lead to a noisy streamline (in red). Since most segments of its path are correct, it still might be 
classified as acceptable by most dMRI signal fitting methods 


In addition to these issues, most dMRI signal fitting methods compute a single 
weight per streamline, which can lead to acceptance of implausible streamlines that 
are erroneous in a small region (cf. Fig. 5). As suggested in [42], this issue can 
be handled by having variable weights along the streamlines. However, this solution 
might come at a cost of numerical instability. Finally, an important aspect to consider 
is the applicability of these methods to diseased brains. For example, it was reported 
in [72] that using SIFT in certain types of illnesses (e.g. brain tumors) can lead to 
wrong conclusions of connectivity changes. 

In [69], we tested the performance of SIFT2 in short synthetic streamlines. First, 
we generated dMRI data with Fiberfox [35] with and without noise from a set of 
straight and bent short streamlines. Second, we added a set of streamlines at a differ- 
ent angle. These added streamlines should ideally be classified as noisy as they do 
not comply with the generated dMRI data. Finally, SIFT2 was run in order to assess 
its ability to separate the ‘signal-generating’ streamlines from the added ones. The 
main result of these experiments is that SIFT2 is able to filter the implausible stream- 
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lines in this simplified problem only for straight streamlines with b-values below 
3000s mm? and low levels of noise. Our hypothesis for this is that dMRI signal 
fitting methods might benefit from longer streamlines for increasing their stability. 
However, as already discussed, stability cannot guarantee a good solution as shown 
in Fig. 5. 

It is worth to note that the filtered tractograms have been reported significantly 
different from the non-filtered ones [45]. However, it is difficult to assess whether or 
not the differences are due to the reduction of noise or to the aforementioned inherent 
problems of this methodology. 

Finally, it is important to remark that global tractography [33, 46] is closely related 
to dMRI signal fitting. Both use the acquired diffusion data to assess the quality of the 
tractograms, one for generating streamlines, and the other to assess the plausibility 
of them. Thus, global tractography shares the same issues of dMRI signal fitting. 
On the other hand, the current dMRI signal fitting methods are more efficient than 
global tractography. 


2.2 Inclusion and Exclusion ROIs 


A different approach is to implement anatomical constraints in terms of using seg- 
mentation masks for filtering out implausible streamlines. 

An intuitive way to employ such masks is atlas-based tractogram filtering. This 
strategy comprises two steps: first, the atlas of streamline bundles is registered to 
the subject data, and second, streamlines that are not fully contained within a single 
bundle mask after registration are filtered out. Despite its simplicity, this method has 
various shortcomings. First, compared to gray matter (GM), registration of WM in 
raw data is more challenging due to the relatively low contrast and the less convoluted 
structure [58]. This makes registration more prone to errors [22]. Thus, different 
methods have been proposed for the specific purpose of WM registration on both 
raw and derived features such as fractional anisotropy (FA) maps (cf. [38] for a 
review of methods). Second, atlas-based approaches are known for not being able to 
model the anatomical variability among subjects very well. Finally, the anatomical 
shape of fiber bundles can change due to illnesses (e.g. brain tumors), making the 
use of atlases difficult in such applications. 

An interesting alternative to atlas-based filtering is to estimate segmentation masks 
of fiber bundles directly from dMRI data. By that, the problematic registration step 
is avoided. TractSeg [67, 68] follows this idea by training a neural network for 
segmenting 72 different fiber bundles. In this approach, fODF peaks are taken as 
input to a 2.5D U-Net architecture for segmentation [48]. Unlike atlas-based methods, 
TractSeg does not require registration and is subject-specific since it only uses the 
acquired data. 

Alternatively, Wassermann et al. [66] proposed the white matter query language 
(WMQL) and its implementation TractQuerier, which can be used to define fiber 
bundles of interest using high-level relationships between GM and WM structures. 
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The method uses any parcellation for locating regions of GM that can be used in the 
query. WMQL can be used for filtering out streamlines not compliant with any of the 
anatomical rules defined for extracting all bundles of interest. Notice that although 
neither TractSeg nor TractQuerier were proposed for tractogram filtering, they can be 
easily adapted for this purpose. Moreover, it is important to remark that TractQuerier 
itself is only a tool to define the mentioned rules. Thus, it can potentially be used for 
tractogram filtering either targeting TPs or FPs. 

It is important to notice that methods using segmentation masks as inclusion or 
exclusion ROIs could be unable to detect certain implausible streamlines that are 
not rejected. That is, defining anatomy solely based on inclusion/exclusion ROIs is 
often only a necessary but not a sufficient condition for determining the plausibility 
of streamlines. Moreover, while some tractography methods do include anatomical 
priors in the formulation (e.g. [26, 54]), in practice, the resulting tractograms usually 
contain false positives. 


2.3 Streamline Geometry or Shape 


In this approach, the shape of streamlines is used for deciding whether or not they 
are plausible. Most tractography methods already include restrictions on the shape 
of streamlines, such as maximum local curvature or minimum and maximum length. 
Moreover, some tractography methods include geometry priors at a higher level [4, 
15]. Since these rules are usually included in state-of-the-art tractography approaches, 
there has been less need for filtering methods implementing this idea. 

A recurrent issue in probabilistic tractography is the existence of streamlines 
with unrealistic loops. In [3], changes in track density are used for detecting and 
removing such loops. Using a more general formulation, it is assumed in [65] that 
the relationship between a streamline and its neighbours should be similar along the 
path. Such relationships are modeled through graphs and are analyzed using spectral 
graph theory. This method is able to remove unrealistic loops while keeping the ones 
that comply with anatomy. 

Recently, the usefulness of streamline geometry for filtering has been shown. For 
example, in ExTractor simple geometry priors are employed to assess streamline 
plausibility [43]. Surprisingly, the authors found that around half of the streamlines 
generated by many state-of-the-art methods do not comply with such priors. A graph- 
based neural network was trained in [1] to reduce the computational cost of ExTractor. 

Additional geometry priors can potentially be useful for tractogram filtering. Geo- 
metrical features extracted from a population could be used to assess the plausibility 
of streamlines at a local level. For example, an atlas of local sheet probability index 
[61], statistical shape models of fiber bundles [15] or an atlas of local orientation and 
curvature [6] could be beneficial for removing false positives. 

Also deep learning-based methods have been proposed for tractogram filtering, 
most of them relying mainly on streamline geometry as criteria. For instance, in 
FiberNet a convolutional neural network is used on the coordinates of the streamlines 
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after a spatial normalization for bundle classification [24]. In [73], the authors extract 
a set of features from the shape of streamlines (named FiberMap) to train their bundle 
classifier. Streamlines not belonging to a known bundle are assigned to an additional 
class. Furthermore, TRAFIC trains a network using distances to five landmarks, 
curvature and torsion per tract as features for filtering [37]. Moreover, DeepBundle 
[30] used a graph convolutional neural network for extracting geometric features 
from the streamlines. Such learned features are then used to assign them to their 
more likely fiber bundle in an end-to-end fashion. The loss function can be designed 
to target false positives. Since these deep learning-based methods use streamlines of 
specific fiber bundles for training, their main target are the TPs in those bundles. 

Similarly to the inclusion and exclusion of ROIs, geometry constraints of the 
streamlines are not necessarily sufficient criteria for deciding the validity of stream- 
lines and must be combined with other priors. That does, however, not apply to the 
mentioned DL-approaches which—depending on the training labels—are able to 
learn a model of plausible streamline geometry. 


2.4 Streamline Similarity and Clustering 


With this strategy, streamlines are clustered in bundles before further analysis. Such 
clusters can be used as surrogates of the underlying structure of WM. Fibers belonging 
to small clusters or that do not share similar properties of bundles of interest can be 
removed. 

The only requirement for using standard clustering algorithms for streamline clus- 
tering is to define a distance metric between streamlines. While proposing distance 
metrics is straightforward, it is more difficult to find the most appropriate one for 
streamlines. Depending on the application, a tractogram can consist of millions of 
streamlines. Thus, it is critical to use efficient implementations. For example, Quick- 
bundles [18] was proposed as a tool for performing clustering very efficiently. 

Once clusters of streamlines are extracted, there are different alternatives for 
performing filtering. For example, Recobundles uses an atlas-based strategy in which 
the clusters are first registered to an atlas of streamline bundles, followed by a pruning 
procedure of streamlines lying far away from the registered centroids [19]. Following 
another strategy, in [74], 800 clusters of streamlines are computed for a number of 
subjects that, after manual curation performed by an expert, are used for creating 
an atlas of streamline clusters. This curated white matter atlas (WMA) is used for 
filtering out streamlines far away from any cluster in the atlas. Following a different 
idea, BundleMAP [27] uses support vector machines on the mean and covariance of 
the coordinates of the streamlines in a bundle to detect FPs. 

Clustering methods based on deep learning have the potential to be computation- 
ally more efficient than classical approaches. In [40], the authors proposed FS2NET, 
a Siamese deep neural network that uses bi-directional long short term memory 
(LSTM) layers for learning a distance measure between streamlines. With this dis- 
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tance, the method can be used to assess if two streamlines should be clustered together 
or not. 

Implausible streamlines tend to follow more erratic paths compared to plausible 
ones. Thus, using clustering is appealing for filtering, since it introduces strong 
requirements on smoothness of streamlines. Moreover, combined with atlas-based 
approaches, they are able to filter both whole brain or partial tractograms. Unlike 
the atlas-based approaches described in Sect. 2.2, the registration is performed on 
tractograms, which tends to be more accurate (cf. [38] for a review of methods). Still, 
the inherent issues of atlas-based approaches might have an impact on the accuracy 
of such methods. 

Since the methods of this subsection are based on bundle similarity, they target 
only certain bundles and, by that, only TPs. 


2.5  Multiapproaches 


From the previous discussion, it is natural to devise methods taking advantage of 
different priors for increasing accuracy. Due to the fact that the research field is 
relatively new, only a few multiapproach methods have been proposed. In this line, 
COMMIT? [50] adds anatomy priors to the original formulation of COMMIT in 
order to target the issues of dMRI signal fitting methods. Another example is anchor- 
constrained plausibility [36], which combines streamline clustering and dMRI signal 
fitting for performing filtering. In [23], FiberNet2.0 has been proposed as an exten- 
sion of FiberNet in which inclusion/exclusion of ROIs are added to the processing 
pipeline. 

We have recently used deep learning for combining two methods: RecoBundles 
and ExTractor [25] using the dMRI signal as the only input of the neural network. 
From our preliminary results, it is not obvious which method should be used as 
gold standard, as the choice of accuracy measurement depends very much on the 
application. Thus, while a perfect combination of priors is not straightforward, from 
our experience in [25], we expect new methods that combine two or more priors to 
perform better on average. 


3 Challenges and Perspective 


Machine learning and in particular deep learning has been very successful in many 
medical image analysis applications in the last few years. Preliminary efforts show 
the potential of this approach also for the specific problem of tractogram filtering 
[1, 25]. However, important challenges for methods following this approach remain 
open. 

As mentioned in [44], there are important general challenges for tractography, 
which also apply for tractogram filtering. Specifically, machine learning and deep 
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learning require large amounts of training data of good quality that are difficult to 
obtain for tractogram filtering. Moreover, the available training data is relatively 
scarce and difficult to combine. Also, inter observer variability of manual annota- 
tions is particularly severe in tractography [47, 52]. Furthermore, it is in general 
questionable if manual annotation of whole brain tractograms would ever become a 
feasible goal. For this reason, the definition of adequate training labels in absence of 
a ground truth or a strong gold standard can be expected to remain the main challenge 
for machine learning-based methods in this field in the future. 

One way of addressing this issue is to combine different methods as automatic 
annotation tools in order to define a gold standard. Following this idea, in [25], we 
proposed a method that builds on top of two methods, namely Recobundles and 
ExTractor, for defining labels. However, this combination is not straightforward. As 
pointed out in Table 1 and in the previous sections, it must be considered that different 
methods assess different characteristics of the tractogram. For example, the rejection 
of a streamline based on geometry priors could have a higher confidence than basing 
that decision on a clustering argument. The reverse is also true: a streamline close to 
an anatomically plausible cluster might be accepted but a streamline compliant with 
a finite number of geometry-based rules could still violate other unchecked rules and 
therefore be implausible. Filtering also depends on the application. If the goal is to 
obtain segmentation masks, geometrical constraints could have a lower value. 

In order to investigate the process of finding a good balance in the combination 
of different automatic annotation tools, we run Tractquerier (TQ), which is an imple- 
mentation of WMQL, Recobundles (RB), TractSeg (TS) and COMMIT (CM) in a 
tractogram of 10 million streamlines computed with ACT [54] for one HCP subject. 
A naive approach to combine the methods would be majority voting. Figure 6 (on the 
left) shows the acceptance rates of streamlines for the testing dataset after perform- 
ing a majority voting with different thresholds. As shown, requiring at least three 
methods to accept a streamline would result in a massive filtering of 95.1% of the 
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Fig. 6 Percentage of accepted streamlines obtained with Tractquerier (TQ), Recobundles (RB), 
TractSeg (TS) and COMMIT (CM) ona 10M tractogram of one HCP subject. Left: Thresholding the 
number of positive labels from the four methods per streamline (majority voting). Right: Individual 
methods 
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dataset. Even with a milder threshold of at least two methods, 79.1% of the dataset 
would be filtered out. While the amount of around 20% of accepted streamlines (i.e., 
two million streamlines in this example) could be enough to fill up the space of the 
WM with an improved TP-to-FP ratio, potential biases of such an ad hoc approach 
would need to be investigated. Also the low agreement for only 50.6% of the stream- 
lines (0.2% for four positive votes and 50.4% for four negative votes) indicates that 
a more sophisticated strategy for the combination of different tools should be con- 
sidered. This could maintain a higher acceptance rate of TP streamlines and by that 
potentially reduce the required number of streamlines in the tractogram. 

Figures 6 (on the right) and 7 show the percentage of streamlines accepted by 
each method as well as their agreement, respectively. As shown, the most restrictive 
method is CM and the most relaxed one is TQ. While the other two methods are in 
the middle, they are also rather restrictive. Moreover, any pair of methods agrees in 
60-80% of the streamlines. From these initial analyses, it is clear that more research 
is needed in order to find better ways of synthesising information from different 
methods for the purpose of tractogram filtering than just simple majority voting. 
Again, the final application must also be considered for assessing the ideal approach. 

In addition to combining different methods, using other prior information can be 
potentially useful for tractogram filtering. For example, including specific microstruc- 
ture information has been useful for tractography [20, 51], which can also be expected 
from tractogram filtering methods. Combining dMRI and functional information 
is promising to understand the mechanisms for brain connectivity [14]. Thus, it 
would be interesting to explore in the future the use of functional data for tractogram 
filtering. 
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4 Conclusion 


Tractogram filtering is a relatively young but very active research area with a high 
potential of development. While the reviewed methods in this chapter show that there 
are a multitude of ways to obtain information related to the plausibility of streamlines, 
there is yet no holistic approach to separate the TP and FP streamlines in a tractogram 
in a fully satisfying way. In our opinion, machine learning-based methods have the 
potential to contribute substantially to tractogram filtering. However, at this moment 
the applicability of supervised approaches is tightly coupled to the proper definition 
of training labels, which is difficult to obtain in the absence of a ground truth. We see 
the combination of different automatic annotation tools, potentially complemented 
with manual annotations from neuroanatomists, as a promising avenue to address this 
problem, while developments are still needed in that line of research in the future. 
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to Brain Anisotropy geto 


Maéliss Jallais and Demian Wassermann 


Abstract This chapter covers anisotropy in the context of probing microstructure of 
the human brain using single encoded diffusion MRI. We will start by illustrating how 
diffusion MRI is a perfectly adapted technique to measure anisotropy in the human 
brain using water motion, followed by a biological presentation of human brain. The 
non-invasive imaging technique based on water motions known as diffusion MRI will 
be further presented, along with the difficulties that come with it. Within this context, 
we will first review and discuss methods based on signal representation that enable 
us to get an insight into microstructure anisotropy. We will then outline methods 
based on modeling, which are state-of-the-art methods to get parameter estimations 
of the human brain tissue. 


1 Accessing Brain Anisotropy Using Diffusion MRI 


1.1 Introduction 


Diffusion-weighted MR imaging is a non-invasive tool used to probe tissue 
microstructure. During a typical acquisition of tens of milliseconds in brain imag- 
ing, water molecules can displace up to tens of micrometers. Diffusion is therefore 
sensitive to a wide range of microstructural and physiological parameters in the 
tissue. The diffusing molecules get restricted by the boundaries of the underlying 
microstructure of tissues. Diffusion anisotropy corresponds to the hindrance of those 
molecules, otherwise free diffusing (i.e. isotropically). Changes in anisotropy have 
been related to brain diseases such as ischemia, multiple sclerosis, trauma, or brain 
tumors [2, 58]. Diffusion anisotropy is therefore considered as a potential biological 
marker for changes in tissue microstructure. A loss of anisotropy can also be the sign 
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of an increasing isotropy, as neurons grow in size. Panagiotaki et al. [57] notably use 
this change to study the evolution of tumor cell size in response to a drug. An under- 
standing of the origin of anisotropy change and a combined study of both anisotropy 
and isotropy can therefore lead to great discoveries on a tissue microstructure and 
its evolution. 

An ultimate goal of a Magnetic Resonance (MR) diffusion theory is then to relate 
the microstructural and physiological parameters quantitatively to the diffusion- 
weighted signal. This task appears to be complicated as deducing those parameters 
constitutes a complex inverse problem requiring careful modeling of the diffusion 
signal over a wide range of diffusion time and diffusion weightings (diffusion weight- 
ings will be further explained in Sect. 2.2). 

Regarding its cellular composition, brain can be decomposed in two main parts: 
white matter and grey matter. The former designates regions that contain mainly 
long-range myelinated axons, which cross the brain connecting different parts of 
grey matter, and relatively few cell bodies. A method to study those connections 
is called tractography and has been well explored during the past few years [28]. 
Grey matter contains mainly cell bodies, connected by neurites, and relatively few 
myelinated axons. Anisotropy exists in both white matter and grey matter and is 
due to the presence of cells with long cylindrical processes (axons in white matter 
and neurites in grey matter). Its presence, or its absence, will provide us with key 
information about the tissue structure at the cellular level. 

Fick et al. dedicated a review on existing diffusion anisotropy metrics [14], which 
includes Fractional Anisotropy (FA) [3], Generalized Fractional Anisotropy (GFA) 
[60], Propagator Anisotropy (PA) [50], Orientation Dispersion Index (ODI) [67], and 
microscopic Fractional Anisotropy (uFA) [33]. We present here a complementary 
approach, which considers anisotropy as a probe for accessing microstructure, either 
through signal representations or tissue modeling. 


1.2 Anisotropy as Reflected by Water Motion 


Particles suspended in a fluid are constantly undergoing small random movements, 
which is known as Brownian motion [42]. The physical process of a steadily spread 
of a substance is called diffusion. Diffusion can therefore be considered a macro- 
scopic manifestation of Brownian motion on the microscopic level. When no barrier 
impedes diffusion preferentially in one direction over another, molecular displace- 
ments are equal in all directions. This is known as isotropic diffusion. However, in 
brain, molecule movements are hindered by cell membranes. Diffusion is then not 
equal along all directions anymore and has become anisotropic. The distance traveled 
by a water molecule depends on its interactions. Certain geometric characteristics 
of the underlying structure at the microscopic level can therefore be inferred from 
the molecule movements [32]. The further a molecule travels during the time of an 
acquisition, the greater signal attenuation we get. The objective is to use this attenu- 
ation to deduce the structure of the medium where the water molecules are trapped 
in. 
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Depending on the diffusion time, i.e. the amount of time between two gradient 
pulses (see Sect. 2.1), the information we get about the structure will be different. 
If the diffusion time was extremely short, only the local intrinsic diffusivity of the 
fluid, i.e. the rate at which particles can spread, would be measured. The hindrance 
effects would only become apparent at longer times. The degree of anisotropy hence 
also depends on the diffusion time. 


1.3 Structural Brain Anisotropy 


Brain tissue is very anisotropic due to the cylindrical shapes of axons and processes. 
Water molecules within those fibers will on average move further along them than 
across them due to their small diameter. Typical axon diameter in humans is of the 
order of 1-10 um [5]. Process diameters in grey matter lie between 0.1 and 15 um. 

The strong anisotropy in white matter due to the axons encouraged its wide study 
over the past decades. The more complex tissue structure and weaker anisotropy in 
grey matter make its study harder. The presence of isotropy in grey matter is partly 
due to the numerous somas whose shapes resemble spheres (see Sect. 4.1). Soma 
diameters range between 20 and 120 um. White matter models need to be adapted 
to account for the presence of somas in order to be applied to grey matter [38, 53]. 
Myelin also appears to modulate the degree of diffusion anisotropy between axons 
and processes (and so between white matter and grey matter), but has a smaller role 
in anisotropy than membrane [6]. 

Note that anisotropy is not only a property of neural fibers. Anisotropy has also 
been observed in liquid crystals, muscles and other tissues, even in fruits and veg- 
etables [5]. The degree of fractional anisotropy (see Sect. 3.1) is however higher 
in healthy neural fibers than in other tissues such as skeletal muscle, kidney, and 
myocardium [11, 18]. 


1.4 Measuring Anisotropy Using Diffusion MRI 


Using Diffusion Magnetic Resonance Imaging (dMRI) as a non-invasive probe in 
human brain, we aim at getting information about its structure. The acquired diffu- 
sion signal is a sum of the diffusion signals coming from each compartment weighted 
by their relative volume fractions [56], and is therefore modulated by the geometry 
of the tissue microstructure. Relevant information to infer from it are soma diame- 
ters, soma and process densities, and diffusivities. Two complementary approaches 
have emerged for extracting these information about the tissue microstructure from 
the diffusion signal: signal representation and tissue modelling (denomination from 
Novikov et al. [48]). 
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Signal representations aim at quantifying parameters and are model-independent 
mathematical expressions. Their parameters do not carry any particular physical 
meaning. Representations can be used to store, compress or compare measurements. 
There is an infinite way to represent a continuous function. One chooses a representa- 
tion according to the need of a particular neuroimaging study [44]. Although signal 
representations are suited for all kind of tissues, they lack specificity and provide 
only an indirect characterization of the microstructure. 

Biophysical tissue models rely on a schematic geometry of the underlying tissue. 
They are pictures representing a physical reality relying on assumptions meant to 
simplify the complexity of a biological tissue. A good model only keeps relevant 
features which characterize the tissue and discards irrelevant degrees of freedom. The 
designed analytical expression is then fit to the diffusion data in order to estimate 
these relevant features of the microstructure. This advantage of providing greater 
specificity and interpretation of biologically-relevant parameters appears to be the 
weakness of the method. Indeed the initial geometric assumption must be chosen as to 
accurately capture all of the features of the tissue that effectively impact the diffusion 
signal in a given acquisition range [48], but we also must be able to mathematically 
solve this inverse problem. Model validations are important because a wrong model 
could lead to wrong interpretations of a physical phenomenon. 

Techniques from these two approaches, signal representations and tissue mod- 
elling, will be reviewed respectively in Sects. 3 and 4. 


2 Diffusion MRI: Introduction to a Non-Invasive Imaging 
Technique 


Nuclear Magnetic Resonance Imaging allows to non-invasively study the brain in- 
vivo, and in particular brain anisotropy, induced by its microstructure. 


2.1 Diffusion MRI Acquisition Sequence 


Consider an MRI acquisition sequence. After slice selection, all the nuclei on this 
plane are precessing at the same frequency. To obtain a diffusion MR image, two 
gradient pulses are added to the acquisition sequence. The first applied pulse is going 
to make the particles go off phase. We then apply a second gradient with the same 
strength in the opposite direction, during the same amount of time. If molecules 
stayed still between those two gradients, they would have all come back to their 
original phase, the two gradients cancelling each other. However, after turning the 
first gradient on, molecules are moving randomly (Brownian motion). After a certain 
evolution time, if molecules are not at the same location, the second gradient causes 
destructive interference, which results in a loss of signal. The further a molecule 
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travels from its initial position during the time between the two diffusion gradients 
along the gradient direction, the greater signal attenuation we get. The ratio between 
the signal obtained with diffusion gradients and the one without them quantifies the 
amount of ongoing diffusion. The objective is then to deduce the structure of the 
medium where the water molecules are trapped in from those signal losses. 


2.2 Mathematical Foundations 


Stejskal and Tanner invented in 1965 the Pulsed Gradient Spin Echo (PGSE) sequence 
[59] to measure diffusion in a specific direction. In this sequence, two opposite 
diffusion gradients are applied during a time ô, separated by a time interval A. The 
diffusion-weighting is globally encoded by the b-value [37], and reflects the strength 
and timing of the gradients used to generate the diffusion weighted images. This 
factor is computed as follow: 


b = y*g75°(A — 8/3), (1) 


where y (MHzT~') is the nuclear gyromagnetic ratio of the water proton !H and 
g is the strength of the diffusion gradient. In the following sections, g encodes the 
direction of the applied diffusion weighting in addition to its strength (g = ||g||), 
and g is the corresponding unit vector (¢ = g/||g||). 

The quantity E(b) = S(b)/So expresses, for each voxel, the attenuation of the 
diffusion-weighted signal along the selected gradient direction, So being the image 
acquired without diffusion gradients. In the absence of restrictions (free diffusion), 
the signal attenuation can be expressed as: 


E(b) =e”, (2) 
with D the diffusion coefficient. 


If ô is assumed to be infinitely narrow, i.e. the diffusion during that time is negligi- 
ble, the signal attenuation can be related to the ensemble average propagator (EAP) 
P(r, T) via a Fourier relationship under the q-space formalism [9, 59]: 


Pans a = Í Per, rede, (3) 


where q is the wave vector and t the diffusion time, which, for the PGSE sequence, 
are expressed as 
q = yôg/2r and t = A — 6/3. (4) 


The diffusion time t expresses the time interval during which spins are allowed 
to diffuse before measurement. By increasing the spatial frequency q = ||q|| it is 
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possible to achieve a higher spatial resolution of P(r, t) in the displacement space 
described by r. 


2.3 Acquisition Strategies 


Experimental parameters, and especially q and t, influence the diffusion signal atten- 
uation along different gradient directions, and therefore the estimation of diffusion 
anisotropy. Ideally, many gradient directions, q-values and diffusion times would be 
required to completely characterize diffusion anisotropy in a tissue. In practice, the 
sampling strategy depends on the application and on the chosen signal representation. 
This way, only one shell of gradient directions and a single b-value are usually used 
in DTI (see Sect. 3.1). Also using only one shell at a higher b-value and more direc- 
tions, are the High Angular Resolution Diffusion Imaging (HARDI) schemes, which 
aim at increasing the angular resolution of the diffusion signal with the intent of 
resolving crossing tissue configurations [61]. Different diffusion-weightings signal 
acquisitions are also needed for some signal representations. In that case, multi-shell 
acquisitions are set up using different q-shells with fixed diffusion time. Each shell 
then represents a collection of samples in the three-dimensional space with the same 
q-value. An optimal spatial coverage is important to measure the diffusion signal 
as efficiently as possible. Expansions have been proposed such that all the acquired 
samples lie on different non-collinear directions [10]. This multi-shell design can 
be extended to t-shells, called gt acquisitions [13] in order to exploit different val- 
ues for both q and t. In that case, a complete q-shell scheme is acquired for each 
desired diffusion time. Ning et al [44] reviews and compares 16 reconstruction algo- 
rithms (single and multi-shells) to help determine an appropriate acquisition protocol 
(number of b-values) and the analysis method to use for a particular neuroimaging 
study. 


2.4 Difficulties 


A main drawback to take into consideration is inherent to the dMRI acquisition 
process. Due to the acquisition device limitations and the mesoscopic size of neurons, 
one voxel, at the macroscopic scale, includes thousands of somas and processes. This 
means that the acquired signal is an average of the signal coming from all those cells. 
Several issues have then to be considered. 

First, the acquired signal in a voxel will be an average of the signal of all the 
diffusing molecules within this voxel, which could correspond to not less than 3000 
axons in white matter. Features that will then be computed from it, such as anisotropy, 
will be an average of all the components in the tissue. One needs to note that every 
tissue is made of several compartments and that the signals from each of these 
compartments where water molecules are present are averaged. Investigations using 
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diffusion-weighted spectroscopy, an imaging technique with increased cellular speci- 
ficity, are also led to try and target specific compartment(s) [38, 54]. This average 
problem leads to a second issue: a small change in anisotropy (or other features) can 
actually reflect greater pathological differences. It means that there needs to be a big 
change in the voxel to be able to detect it in the acquired diffusion signal. Third issue 
is that anisotropic cellular elements might be considered as isotropic due to the tree 
pattern of processes within grey matter [26] or to crossing fibers in white matter. At 
least, as expected from an acquisition, the signal is noisy. Low concentration of water 
molecules in some tissue (and thus long scan times) can lead to a poor signal-to-noise 
ratio. 

In addition to those issues, we must recall that the spacing between axons, axon 
diameter, myelin thickness, etc are all also variables, even within the same tract, 
which adds to the complexity of the problem. The barriers to diffusion have also not 
a simple nor regular geometry. The correspondence between the biological features 
of the tissue and the non-invasive diffusion measure is therefore not straightforward. 


3 Quantifying Anisotropy via Signal Representation 


Signal representation is an indirect method that aims at describing the diffusion signal 
with no assumptions about the underlying structure. It can therefore be applied to 
healthy or diseased tissues. Several methods are described, with an emphasis made 
on the cumulant expansion, which is the most widespread signal representation. 


3.1 Cumulant Expansion 


Common signal representations are based on the cumulant expansion [39, 65], which 
corresponds to a development of the logarithm of the signal in polynomials up to a 
given order in b: 


n() = -bD + l DY K +... (5) 
So 6 


where D is the diffusion coefficient and K the kurtosis. This formula can also be 
written in the tensor form: 
S) 


In ~ bC) gi Bi, +D CÊ 8n Big — (6) 


where C are the cumulant tensors, and g is the direction of the applied diffusion 
weighting (see Sect. 2.2). Note that Einstein’s convention of summation over repeated 
indices is used here. 


178 M. Jallais and D. Wassermann 


An expansion in moments, which corresponds to a Taylor expansion of the signal, 
is also possible. While expansions in moments and in cumulants are mathematically 
equivalent, for a similar order truncation at some fixed (low) order, the cumulant 
series provides a more accurate estimation of the dMRI signal than a moment expan- 
sion. Moment expansion is more optimal for analytical treatments because contri- 
butions from different tissue compartments add up. A combinatorial relation exists 
between the two expansions [40, 65]. Computing the cumulant tensors and convert- 
ing them into moments is promoted to be the most numerically stable methodology to 
adopt [49]. 

One of the most popular MRI techniques in brain research as well as in clinical 
practice is Diffusion Tensor Imaging (DTI) [3], based on the cumulant expansion up to 
the first order in b. This technique is valid for low diffusion weighting (b « (DK)~ 1), 
Note that this technique does not assume that the medium is homogeneous with 
unrestricted diffusion (K = 0), which appears to be not true for most biological 
tissues, but that it follows a Gaussian law when b &« (DK)~!. 

Using tensor decomposition, three eigenvalues (A;, Az and à3) reflecting axial 
and radial diffusivity of molecules within fibers and in the extra-cellular space are 
computed (see Fig. 1). The difference between those two diffusivities enable to 
define variables such as Mean Diffusivity (MD) (the average of all the eigenvalues) 
or Fractional Anisotropy (FA), defined as follow: 


A _ ie V1 =A)" + Qa = As)" + As =A)? 
rms(A) 2 JENER 


Those two measures are complementary, as they bring different information to 
the comprehension of a tissue (Fig. 2). Hofstetter et al. [19] used MD to hypothesize 
the presence of bigger cells in the brain after a learning session. Beaulieu et al. also 
investigated anisotropy in the human brain grey matter using DTI [4]. 

Diffusion Kurtosis Imaging (DKI) goes beyond DTI and its first order expansion 
by also estimating the kurtosis of the diffusion probability distribution function [25]. 


(7) 


Fig. 1 Isotropic diffusion in somas can be modeled by a sphere (left). Anisotropic diffusion in 
neurites can be represented by an ellipsoid reflecting axial (A;) and radial (A2 and 43) diffusion. 
This image has been inspired by the book chapter written by Christian Beaulieu [4] 
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FA = 0.12 FA = 0.46 FA = 0.82 
A, = 0.80 à, =1.10 A, =1.60 
A, = À, = 0.65 A, = À, = 0.50 A, =A, =0.25 


Fig. 2 Mean diffusivity (MD) and fractional anisotropy (FA) are two complementary measures. 
Here are three examples of ellipses ranging from isotropic to anisotropic that have the same mean 
diffusivity (0.7 x 107° mm?s~!). This image has been inspired by the book chapter written by 
Christian Beaulieu [4] 


The kurtosis quantifies the non-Gaussianity of a distribution. The information that it 
provides is complementary to DTI metrics. Fitting the kurtosis tensor significantly 
improves the accuracy of the diffusion tensor estimation [64]. In a same way, extend- 
ing the series to the sixth order cumulant (in b?) also increases the accuracy of the 
kurtosis estimation, albeit with a penalty on precision. 

In order to estimate the six independent components of the diffusion tensor, the 
minimal required data is one b = 0 (unweighted) image and six non-collinear direc- 
tions on a single diffusion weighting, or “shell”. The additional estimation of the 15 
independent components of the kurtosis tensor requires a minimal acquisition of one 
b = 0 image and one or two nonzero shells with 15 non-collinear gradient directions, 
so that a total of 22 diffusion-weighted images are acquired [64]. The choice of the 
shell b-values is a trade-off between accuracy and precision. The b-values should 
be as low as possible to respect the validity of the cumulant expansion, but slightly 
higher values enable to limit the impact of noise [12]. Jelescu et al. [21] suggest a 
typical value around b = 1 ms um~? for DTI and 2 ms ym~? for DKI in vivo. For 
further details on the optimization of acquisition parameters for precise measurement 
of diffusion in anisotropic systems, we invite the reader to have a look at the work 
of Jones et al. [31]. 


3.2 Other Representations 


Yablonskiy et al. [66] hypothesize that the acquired diffusion signal is a sum of 
signals originating from many spin packets, present in different cell types, at different 
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positions. Each spin packet having then different trajectories and facing different 
hindrances, they make the assumption that they all have a different apparent diffusion 
coefficient (ADC). Hence, they introduced a distribution of diffusion coefficients 
p(D), and expressed the diffusion signal as following: 


2 = Í j p(D)e Pd D (8) 


Theoretically, the distribution of diffusion coefficients can be estimated using the 
inverse Laplace transform. In practice, some functional form needs to be assumed 
for o(D) due to the mathematical ill-conditioning of the inverse Laplace trans- 
form. In addition, a very strong diffusion weighted regime is needed for the esti- 
mated distribution to accurately reflect the tissue distribution of diffusion coefficients 
[34, 47]. 

Jian et al. [29] propose a statistical method to infer connectivity patterns based on 
the characterization of the water molecule diffusion by a continuous distribution of 
diffusion tensors. They described the MR signal attenuation as the Laplace transform 
of this probability distribution defined on the manifold of symmetric positive-definite 
tensors. Combined with a spherical deconvolution approach, displacement probabil- 
ity functions and distinct fiber orientations can be estimated in each voxel ina HARDI 
dataset. 

The multi-shell Mean Apparent Propagator (MAP)-MRI method, as proposed by 
Ozarslan et al. [50], expands the signal using harmonic oscillator basis functions. 
It indeed represents the diffusion-weighted signal by an anisotropic Gaussian mod- 
ulated by a series of Hermite polynomials. This method allows the estimation of 
three-dimensional EAP, where both restricted (non-Gaussian) diffusion and crossing 
axons can be represented. However, according to [44], this method fails to estimate 
crossing angles correctly. The strength of the method resides on its capacity to accu- 
rately estimate diffusion properties such as return-to-origin probability, and mean- 
squared displacement. The propagator anisotropy (PA) metric was derived from this 
method, which is a measure of dissimilarity between the reconstructed EAP and its 
closest isotropic approximation EAP. 

Hanyga et al. [17] proposed a new space-fractional diffusion model based on an 
anomalous anisotropic diffusion equation that preserves posivity. This method seems 
well-suited for applications to DTI [42]. 

Other representations exist, but have not been included in this chapter. 


3.3 Limitations 


The validity and therefore the usefulness of the cumulant expansion depends on its 
convergence towards the acquisition signal, characterized by the convergence radius 
b. [35]. If b < b., then the series can be approximated using a couple of low terms 
in Eq. 6, higher order terms being flooded by the noise, i.e. small contributions to 
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the signal can not be decoupled from the noise in experimental data. The number 
of parameters to estimate is then reduced, but a good accuracy does not assure its 
validity. Otherwise, ifb > b,, the series in Eq. 6 diverges which means that the model 
function cannot be reduced to a polynomial. A good quality fitting gives then more 
credit to the underlying model. 

Hutchinson et al. [20] compare the DTI, DKI, MAP-MRI and NODDI (see 
Sect. 4.4.1) methods in different experimental conditions to study the influence of 
noise and sampling (among others) on parameter estimations. All methods proved to 
be influenced by the acquisition parameters such as the b-values, the resolution, the 
SNR and the diffusion time. The need of DKI to fit a higher order tensor explains its 
high sensitivity to noise. 

Regional issues are also to be noted, related to crossing fibers, which can be 
detected as isotropic zones [1]. Indeed, several diffusion directions are possible in 
that case. The angular resolution needs to be high enough and the model designed 
to take this particular case into account. 


4 Biophysical Modeling to Measure Anisotropy 


This second approach is based on a biophysical model designed for a particular 
tissue geometry. This model is fit to the diffusion signal acquired, which allows the 
estimation of the relevant parameters of the microstructure. While it can provide a 
greater specificity of biological parameters, the design of the model remains difficult, 
as it needs to accurately capture all the features that effectively and substantially 
impact the diffusion signal in a given acquisition range (the coarse graining problem, 
see [46]). 

Another big challenge of this approach comes from the number of unknowns to 
estimate after the definition of all effective parameters. To estimate them all we would 
need a lot of different b-values. This is unfeasible in clinical applications, because 
first the gradients used in clinical MRIs are not strong enough, and secondly it would 
require a patient to stay in the MRI device for a very long time. Some methods rely 
on constraint to bypass this problem, as presented in Sect. 4.4. 


4.1 Multi-compartmental Model 


Tissue in the brain can generally be decomposed into four compartments. The first one 
corresponds to the somas, which are the central part of the neurons, mainly present in 
grey matter. Glial cells are also comprised in this compartment, as done by Palombo 
et al. [54]. However, their possible high exchange rate with the extracellular space 
(ECS) is still a matter of discussion and this argument would argue in favor of their 
better modeling in the ECS compartment [15]. Somas can be modeled as spheres 
of different diameters. Neurites, the second compartment, connect those neurons 
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together, either in short distances in grey matter (they are called dendrites), or long 
distance connections in white matter (axons). The diffusivity across the processes is 
considered zero due to the restriction implied by the fixed small diameter. Processes 
can therefore be modeled by cylinders with zero-radius (“sticks”) (see part Sect. 4.2 
below). The orientation of a collection of processes within a voxel is characterized by 
an orientation distribution function (ODF) [60]. The third compartment corresponds 
to extra-cellular space (ECS) and is modeled as Gaussian anisotropic. The last one 
is the cerebrospinal fluid (CSF), which could contribute if a voxel contains part of 
a ventricle, and corresponds to free diffusing molecules. It is hence modeled as free 
diffusion. 

The acquired water signal originates from these four compartments and are 
weighted according to their relative signal fraction f: 


S(b) = Feomas i Ssomas (b) + Seurites g Sneurites (b) + Secs ` SECs (b) + fcsF d Scsr (b), 

(9) 
with Jsomas + Freurites T Secs T fesF = 1. Remark that Fsomas> Fneuritess SECS and ScsF 
are not the relative volume fractions due to the T2 differences between the compart- 
ments. In the following models presented, a combination of those compartments is 
used to model particular tissues and keep only the relevant compartments. Note that 
a common assumption is made that the exchanges between the compartments can be 
neglected at the time scales of clinical dMRI, at least in white matter. The estimation 
of exchange rate in vivo is challenging and more investigations are needed to validate 
this hypothesis in white and gray matter. 


4.2 Neurites as Sticks 


Neurites have been modeled by zero-radius impermeable cylinders, characterized 
by their longitudinal diffusivity, the transverse diffusivity being considered zero. 
These neurites are called “sticks” and correspond to the most anisotropic Gaussian 
compartment possible [7, 26, 36]. 

The intra-neurite response function, i.e. the diffusion signal from water inside a 
stick of diffusivity D, pointing in the unit direction ñ, is defined as: 


GB, b) = eri, (10) 


with g being the unit gradient direction of the measurement. It is determined by 
cos @ = g- ñ, where @ is the angle between n and ĝ. 
The signal, after being isotropically averaged over multiple gradient directions g, 
is the following [8, 24, 30]: 
Sx p-b, (11) 


with B = Jt : focurites/ (DID. 
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In brain tissue, at sufficiently large b-values, the extra-axonal space signal is 
exponentially suppressed, its diffusivity being non-zero in any direction. The only 
remaining signal in white matter comes from the axons (Sheurites in Eq. 9), and follows 
the power law from Eq. 11 [41, 62]. This equation captures the very anisotropy of 
white matter. 

Veraart et al. [63] recently proved that the radius of the axons can be estimated for 
very high b-values, where the transverse diffusivity is not considered null anymore. 
The direction-averaged DWI signal then follows the following law: 


5 ~ Be Pa . pol? (12) 


Such law however does not hold in gray matter, which indicates that white mat- 
ter and grey matter require different models in order to accurately capture their 
microstructure. Several hypothesis have been elaborated to explain the different 
behaviour of gray matter DWI signal. McKinnon et al. [41] and Veraart et al. [63] 
suggest that an increased permeability in cell membranes of neurites in gray matter 
might be the cause of an increased exponent, while Palombo et al. [52] advocate 
the abundance of cell body in gray matter. Ozarslan et al. [51] suggest that curvy 
projections, along with longer pulse duration, lead to a disappearance of the b~!/2 
decay. 


4.3 Standard Model of Diffusion in Neural Tissue 


The measured diffusion signal in brain white matter is a sum of anisotropic com- 
partments. It can be modeled as a convolution between a response kernel K from a 
perfectly aligned fascicle pointing in the direction n and the fiber orientation distri- 
bution (ODF) Pi) normalized to f daP(n) = 1. 


S(b) = f PAK(b, $ - û)dû, (13) 
|al=1 


& being defined in Sect. 2.2. 
In the case of white matter, the kernel can be written as: 


An Il L\g2 
-bD}-b(D!-DŁ)£ + fosp e~PPese 


(14) 


K(b, £) = So a + (1—f — fesr)e 


with € = g - ñ. Those exponential contributions correspond to the intra-axonal space 
modeled by a stick compartment (Eq. 10), the extra-axonal space modeled by an axi- 
ally symmetric Gaussian compartment with transverse and longitudinal diffusivities 
D+ and D!!, and the cerebrospinal fluid (CSF) compartment. All those compartments 
are represented in Fig. 3. This decomposition has been widely used in white matter 
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fiber 


Fig. 3. Standard Model of diffusion in neuronal tissue. Two-compartment model (intra- and extra- 


neurite spaces) described by 4 independent parameters: f, Da, D! and DŁ} and a fiber orientation 
distribution P(ñ). This figure is reproduced from Novikov et al. [49] 


by the community. As a consequence, Novikov et al. suggested to call it the Standard 
Model (SM) [46]. For the sake of reference, we will also refer to it as the SM in this 
chapter. 


4.4 Standard Model Parameter Estimation Using Constraints 


In the previous sections we presented the SM of diffusion in neural tissue as a sum 
of anisotropic Gaussian compartments, as defined by Novikov et al. [46]. We will 
now introduce some methods based on the SM that rely on constraints to overcome 
the challenge of estimating many biological parameters of interest. 


4.4.1 Neurite Orientation Dispersion and Density Imaging 


In order to reduce the number of parameters that need to be estimated, Zhang et al. 
[67] proposed to impose restrictions on the intrinsic diffusivities. They introduced a 
method called Neurite Orientation Dispersion and Density Imaging (NODDI), which 
relies on a three-compartment SM (intra-axonal space, extra-axonal space and CSF), 
described by seven parameters: volume fractions fintra and fiso, diffusivities D! ; 
D!, DŁ and Diso, and the orientation dispersion modeled by a Watson distribution 
of concentration parameter «. By fixing the diffusivities to the following values: 


D! = D! = 1.7 um?/ms (15) 
Dy = (1— fima) D! (16) 
Diso = 3 um? /ms (17) 


only the two volume fractions and the orientation dispersion need to be estimated. 
Although this method allows to estimate the parameters, the validity of those 
constraints need to be questioned. To begin with, if we admit that the equalities are 
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correct, they imply that a small deviation from the fixed values, as occurs in cerebral 
ischemia, will induce a non-negligible bias in the other parameters estimation, leading 
to false interpretations. However, studies using Diffusion weighted spectroscopy MR 
which can quantify the diffusion of specific metabolites (e.g. [55]), suggest, through 
the study of metabolites specifically found on different sections of the neurons and 
extra cellular tissue, that such diffusivity is not constant across the whole brain. 
Whether and how these findings can be used to shed light on water diffusion in the 
brain, is an open question. 

Another drawback of this method is that it leads to indetermination, which means 
that NODDI returns one possible result among a multiplicity of mathematical solu- 
tions by fixing D! = D! [23, 49]. If we consider the case where all the parameter 
constraints are released and the CSF compartment neglected (called NODDIDA [22], 
which stands for NODDI with diffusivity assessment), two distinct solutions to the 
estimation problem exist: D! > D! and D! < D! (see Figs. 8 and 9 in Jelescu et al. 
[23]). Both solutions lie within biologically plausible ranges, and determining which 
solution is biologically correct is an active field of research, although most studies 
are suggesting D! > DL. At least, the tortuosity approximation that relates D+ and 
D! has been invalidated in the case of tight packings of axons [45]. 


4.4.2 White Matter Tract Integrity Metrics 


Another approach to estimate the relevant features of interest in a tissue proposes 
to relate the scalar parameters to the DKI components. Called White Matter Tract 
Integrity (WMTI) [16], it is a two-compartment SM that relies on the assumption 
that sticks are highly aligned within a voxel. 

The tissue is described as a sum of two Gaussian compartments (intra- and extra- 
axonal space, Eq. 9 with fsomas = 0 and fcsp = 0), where axons are modeled as 
sticks embedded in a Gaussian anisotropic extra-axonal medium. Each compartment 
is characterized by a tensor (D, and D,) derived from the kurtosis tensors D and K. 
In any direction j: 

Dj = Fintra Daj + el = Finwa) Dej, (18) 


(Dej — Daj) 


2 
Dij 


K; = 3 fintra i ad finra ) (19) 


We retrieve the two possible mathematical solutions mentioned before, as demon- 
strated by the square in Eq. 19. The solution chosen in this method is D! < DI, 


which leads to: 
K max 


K max + 3? 


K;- intra 
Dz; = Dj [i+ parmi (21) 


Fintra = (20) 
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K; — intra 
Daj = Dj|1—,| pe l 22) 
intra 


Although WMTI enables to capture the changes of diffusivities, it has two main 
limitations. First, this approach is limited to regions of highly aligned single fiber 
bundles, which are only present in some white matter regions. Jespersen et al. sug- 
gested a method that alleviates this assumption by assuming a Watson distribution 
of the axons (like in NODDI) [27]. Second, as it relies on the DKI decomposition, 
this method is only restricted to the low b-value regime, which could lead to some 
bias. 


4.5 Lemonade 


As explained in Sect. 4, estimating both compartment diffusivities and orientation 
dispersion of neurites simultaneously is problematic and tends to be biased. Some 
methods suggest fixing some parameters such as NODDI or to limit its application 
to coherent fibers only as WMTI to work around these problems. Releasing these 
constraints necessitates to estimate a larger number of parameters. 

A very recent method in white matter estimates the scalar parameters of a two- 
compartment kernel separately from the ODF without any constraints. The method 
developed by Novikov et al. [49] is based on the modeling of the diffusion signal 
as a convolution of the ODF and the response kernel from a perfectly aligned fiber 
segment, as presented in Sect. 4.3. It can be decomposed into two steps. A first 
step solves an algebraic system of equations that relates the kernel parameters to 
the signal moments for low b-values. This part was called LEMONADE, which 
stands for Linearly Estimated Moments provide Orientations of Neurites And their 
Diffusivities Exactly. It requires at least 3 non-zero b-shells inferior to 2.5 ms um~? 
and returns estimates for fintra, D!, DL, DŁ and p2 = e, which gives an 
estimate of the orientation dispersion. In a second step, a rotationally invariant energy 
function of the system is minimized exploiting all available data and using the first 
estimates as initialization values. 

This method emphasizes the existence of the two mathematical solutions as intro- 
duced before, and shows that, in principle, the degeneracy can be avoided using 
measurements up to the 3rd order of b-values. However, due to noise in the data, the 
solution selection remains difficult in practice and individual validation should be 
carried out. 

The assumptions made in this approach are, as in the other methods previously 
presented, the existence of only two compartments, the uniformity of diffusivities 
across all axons in the voxel, and axial symmetry of the kernel. These assumptions 
are also the limitations of the model used. Validation in the case of pathological tissue 
also needs to be investigated. 
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We refer the reader to Jelescu and Budde’s review on the accuracy and validation of 
biophysical parameters of different diffusion models in white matter, which includes 
the ones presented before [21]. 


5 Summary and Above 


We have shown two main approaches to describe microstructure anisotropy using 
diffusion MRI: signal representation and biophysical modeling. While the former are 
general and make no assumptions about the underlying tissue, models are designed 
for a particular tissue and therefore provide greater specificity and interpretation of 
the estimated biological parameters. The difficulties in modeling reside in accurately 
capturing the features that effectively and substantially impact the diffusion signal in 
a given acquisition range, and being able to correctly fit the model (inverse problem). 

Anisotropy provides great insight into a tissue structure, and its evolution can 
enlighten the progression of certain pathologies. The presence of isotropy must not 
be neglected either, as it can be a great marker of other microstructures, such as in 
grey matter where it denotes the presence of somas. 

Although great progresses have been made during the last decade, some questions 
remain unresolved. To cite a few, we can wonder to which extent we can consider 
compartments as non-exchanging. Diffusion time, brain region and myelination of 
the tissue will most likely impact the answer of this question. Can we also come 
up with methods less sensitive to the signal-to-noise ratio or a way to disentangle 
thermal noise and artifacts from the signal of interest? 
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Abstract Diffusion-weighted magnetic resonance imaging (MRI) is sensitive to 
ensemble-averaged molecular displacements, which provide valuable information 
on e.g. structural anisotropy in brain tissue. However, a concrete interpretation of 
diffusion-weighted MRI data in terms of physiological or structural parameters turns 
out to be extremely challenging. One of the main reasons for this is the multi-scale 
nature of the diffusion-weighted signal, as it is sensitive to the microscopic motion 
of particles averaged over macroscopic volumes. In order to analyze the geometrical 
patterns that occur in (diffusion-weighted measurements of) biological tissue and 
many other structures, we may invoke tools from the field of stochastic geometry. 
Stochastic geometry describes statistical methods and models that apply to random 
geometrical patterns of which we may only know the distribution. Despite its many 
uses in geology, astronomy, telecommunications, etc., its application in diffusion- 
weighted MRI has so far remained limited. In this work we review some fundamental 
results in the field of diffusion-weighted MRI from a stochastic geometrical perspec- 
tive, and discuss briefly for which other questions stochastic geometry may prove 
useful. The observations presented in this paper are partly inspired by the Workshop 
on Diffusion MRI and Stochastic Geometry held at Sandbjerg Estate (Denmark) in 
2019, which aimed to foster communication and collaboration between the two fields 
of research. 


T. Dela Haije (Ex) 
Department of Computer Science, University of Copenhagen, Copenhagen, Denmark 
e-mail: tom @di.ku.dk 


A. Feragen 

Department of Applied Mathematics and Computer Science, Technical University of Denmark, 
Lyngby, Denmark 

e-mail: afhar@dtu.dk 


© The Author(s) 2021 193 
E. Ozarslan et al. (eds.), Anisotropy Across Fields and Scales, 
Mathematics and Visualization, https://doi.org/10.1007/978-3-030-56215-1_9 


194 T. Dela Haije and A. Feragen 


1 Introduction 


Diffusion-weighted magnetic resonance imaging (MRI) [22] is one of the few imag- 
ing modalities that is capable of mapping the immensely complex micro-structural 
architecture of the human brain in a non-destructive manner. This is achieved by 
applying a specific sequence of diffusion-sensitizing magnetic field gradients during 
the MRI acquisition, producing a signal with a decay rate that is dependent on the 
relative mobility of water molecules in the tissue [30]. As the overall mobility of the 
molecules is decreased by the presence of any material barrier, the acquired signal 
effectively provides an indirect probe of the ambient structure. At the micrometer 
length scales accessible in current scanners, the dominant barriers to the diffusing 
molecules in the brain are the fiber-like neurites that transmit information between 
different regions of the brain [3], and whose properties are (naturally) of profound 
importance in neurology. Aside from the anticipated and present clinical value of 
this modality, diffusion-weighted MRI also stands to provide unique information 
about the evolution, morphogenesis, and function of the brain, already being pur- 
sued through the tracking of macroscopic fibers in tractography [2, 4, 20]. The 
challenge faced by the diffusion-weighted MRI community is to identify the rele- 
vant structural parameters determining the signal decay, and—to the extent that this 
is possible—to invert the relation between them. This is where we believe stochastic 
geometry could play a role. 

Stochastic geometry [29] is an area of statistics that provides modeling and infer- 
ence techniques for complex spatial objects whose structure can be described effec- 
tively as random patterns. One of the first cases studied in what is now called stochas- 
tic geometry—the problem of stereology [1]—included the inference of geometric 
properties of 3D objects from their intersections with a small number of 2D planes. 
This topic gained traction in the 1960s, as its solutions alleviated the challenging and 
computationally demanding task of actually computing 3D reconstructions. Stereol- 
ogy found a wealth of applications ranging from geology to for example microscopy 
of neuroanatomy. Since then stochastic geometry has evolved into a mature field of 
mathematics that offers a rich toolbox of rigorously developed techniques, includ- 
ing models with potential relevance for diffusion-weighted MRI such as random 
tessellations and fiber processes. Skimming the stochastic geometry literature one 
quickly comes across a number of concepts that have obvious analogues indepen- 
dently developed in the diffusion-weighted MRI literature, and yet the two research 
domains have had very limited contact so far. 

This chapter is written from the perspective of diffusion-weighted MRI as a field 
working on a set of challenging modeling problems, focusing on the potential utility 
of stochastic geometry in addressing them. Our aim is not to provide an exhaustive 
overview of relevant theory in either field, but simply to highlight some well-known 
results where the conceptual links between the two become apparent. A compre- 
hensive introduction to stochastic geometry can be found in the book by Stoyan 
et al. [29], while the books by Jones [16] and by Johansen-Berg and Behrens [15] 
provide a good entryway to diffusion-weighted MRI. 
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To initiate an exchange of ideas between these two fields, the first Workshop on 
Diffusion MRI and Stochastic Geometry was co-organized by the authors and Eva B. 
Vedel Jensen at Sandbjerg Estate (Denmark, January 20-24, 2019). The observations 
presented in this chapter partly inspired this workshop, but the chapter likewise builds 
on the valuable discussions held at the lively and interactive workshop. We are not 
aware of any previous works pointing out the parallels described here, although some 
of them will undoubtedly have been noticed before. 


Overview 


The molecular dynamics relevant for diffusion-weighted MRI are conveniently 
described by a displacement probability density function—the diffusion propaga- 
tor P(r, t). The propagator P(r, t) represents the probability of a displacement r at 
a diffusion time t, and the diffusion-weighted signal is related to the characteristic 
function S of the propagator given by 


S(q) = [are t) dr, a) 


where qr denotes the inner product between q and r. The Fourier parameter q can 
be considered an experimental parameter, determined in practice by the diffusion- 
sensitizing gradients of the acquisition. 

In the following sections we give examples of connections to stochastic geometry 
for three different ‘limiting regimes’ of the diffusion-weighted signal, where expres- 
sions for the relevant molecular dynamics can be simplified significantly. Sections 2 
and 3, which feature the short and long diffusion time limits respectively, can be 
well-understood in terms of the time-dependent diffusion coefficient D(t), which 
represents the mean squared molecular displacement at a time t. This diffusion coef- 
ficient appears as the first non-trivial coefficient in the Taylor series of log S, the 
cumulant expansion. ! 

log S(q)  —D(t)q? +... (2) 


From this expansion it can already be gathered that results based on the diffusion 
coefficient are mostly applicable when q is relatively small—i.e., when the gradi- 
ents are weak enough for the D-dependent term in this expansion to dominate—and 
in Sect. 4 we consider instead the limit where the gradient strength parameter ~ q 
becomes large. In this regime the diffusion coefficient no longer provides an ade- 
quate vehicle for the description and analysis of the diffusion process, and we have 
to rely on other descriptors. Where necessary, additional details on these concepts 
will be given in the text, although technicalities will be skipped in favor of acces- 
sibility. To simplify the exposition further, we will make implicit use of the narrow 


'We restrict ourselves in Eq. (2) to the one-dimensional case, to keep technicalities to a minimum. 
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pulse approximation [6, 27] throughout. The paper concludes with an outlook on the 
possible future uses of stochastic geometry in Sect. 5. 


2 Specific Volumes and the Short-Time Limit 


...the time-dependent diffusion coefficient D(t) of mobile molecules confined in pores or 
cells carries information about the confining geometry. At early times, [a perturbative expan- 
sion of D(t)] gives, irrespective of details, the pore surface to volume ratio ..., which is a 
measure of microscopic length. 

Sen (2004) 


The first regime we consider is the short-time limit, described in the seminal works 
by Mitra, Sen, and others [18, 19, 26], and with important insights dating back to 
Kac [17]. The situation analyzed in these works is the diffusion of molecules in the 
neighborhood of obstructive geometrical features, Fig. 1. At the boundaries between 
the diffusive medium and the geometry, the diffusion is prescribed by boundary 
conditions that can depend on e.g. the permeability, leaving the spatially averaged, 
time-dependent diffusion coefficient D(t) to be solved. While this is a very challeng- 
ing problem for any non-zero, finite time f, the limiting behavior for t —> 0 can be 
expressed in terms of a relatively small set of practically useful structural parameters. 
As D(t) can be measured in the scanner, we can use diffusion-weighted measure- 
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Fig. 1 A schematic showing a particle (black solid line) diffusing in a volume V, marked by the 
blue hatch pattern. The gray boundary S acts as a barrier to the diffusion, resulting in a decreased 
time-dependent diffusion coefficient D(t) compared to the free diffusion coefficient Do. At short 
times only particles in a small neighborhood of the boundary (within a distance ~ ./ Do t) are 
significantly impacted by this effect, resulting in a characteristic decrease of D(t) proportional 
to the ratio between the surface area S and the volume V. Note that the time-dependent diffusion 
coefficient measured in diffusion-weighted MRI is acquired by averaging over volumes much larger 
than the typical diffusion length ~ Do t 
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ments to obtain estimates of these parameters. The emergence of these structural 
parameters in the diffusion-weighted signal can be understood as follows. 

In the limit t — O the diffusing molecules do not have enough time to interact 
with the geometry, and the diffusion coefficient naturally approaches the medium’s 
free diffusion coefficient D(O) = Do—the diffusion coefficient of the medium in the 
absence of any geometry. At times close to the limit, the particles move a typical 
distance in the order of ~ Do t (by virtue of the definition of D in terms of the mean 
squared displacement), and so roughly speaking only the fraction of particles within 
some distance ~ ./Do t can ‘see’ the geometry. As the geometry essentially acts as 
a barrier to the diffusion, the time-dependent diffusion coefficient decreases from 
its free diffusion limit, and this decrease is more significant if a larger fraction of 
the total number of particles can interact with the boundaries. For diffusion times 
approaching 0, this fraction becomes exactly proportional to the surface area of the 
geometry. Formalizing this notion, Mitra et al. [19] then showed that the first order 
correction describing the approach of the limit becomes proportional to the surface- 
to-volume ratio S/V of a smooth geometry, cf. Fig. 1, according to 


4 S 
DW) ~ Do- zg yY ot Do (t > 0), (3) 


where d is the spatial dimension. From this relation, the surface-to-volume ratio can 
be estimated reliably from the diffusion-weighted measurements [13]. The order 
t terms in this expansion depend to varying degrees on the boundary condition 
parameters such as the permeability, as well as on the average curvature of the 
geometry, while higher order terms depend on even more intricate details of the 
environment. 

Although the complex interactions between particles undergoing random dis- 
placements and their surrounding structures have not been considered in stochastic 
geometry as such—this problem is closer to mathematical physics [5]—the surface- 
to-volume ratio uncovered in Eq. (3) is a commonly estimated quantity in for example 
stereology. The surface-to-volume ratio is also referred to as the surface density or 
as the specific surface area in stereology and stochastic geometry. An example of a 
question in this scenario for which stochastic geometry could be helpful is: “what is 
the smallest observation window (voxel) that has an acceptable error when estimating 
a given characteristic?” [29, Sect. 6.4.6]. 


3 Stationarity and the Long-Time Limit 


That is, we assume here that the voxel is statistically homogeneous. This macroscopic uni- 
formity allows us to go from averaging over the contributions from all parts of the system ... 
to ensemble averaging over all disorder realizations, leading to the description of the signal 
in terms of the statistical properties embodied in the correlators ... 


Novikov and Kiselev (2010) 
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In the short-time limit of the previous section, particles could not explore their sur- 
roundings outside a vanishingly small window, leading to the simple relation between 
the surface-to-volume ratio and the time-dependent diffusion coefficient. In the long- 
time limit, on the other hand, we can assume that particles in fact see all the sur- 
rounding structures. Under fairly general conditions, the diffusion process in this 
limit can still be described in terms of a single diffusion coefficient—the limiting 
coefficient Dæ = lim;_,.. D(t). While determining Də for a given sample is very 
difficult, Novikov et al. [21, 23, 24] showed that once again there are basic geomet- 
rical properties of the structure that completely determine the limiting behavior of 
D(t). In this case, the diffusion coefficient approaches its limit as 


D) ~ Do tect” (t —> ov), (4) 


where c is a constant, and where the exponent 0 = (p + d)/2 is determined by the 
spatial dimension d and a structural exponent p. The structural exponent p is the 
exponent of the limiting power law behavior of the structure correlation function, 
which roughly speaking quantifies the presence of long-range correlations in the 
diffusion barriers. While the constant c varies significantly between different config- 
urations of barriers, we know of only a few possible values for the exponent p, with 
many vastly different configurations of barriers producing the same critical exponent 
in experiments. The structural exponent p effectively distinguishes between different 
structural universality classes [23]. 

A central concept in the cited works by Novikov et al. is that because the dif- 
fusing particles can explore their entire surroundings, the exact configuration of the 
barriers becomes irrelevant. Consequently, one can theoretically replace the compli- 
cated original structure with a simpler-to-analyze effective medium, permitting the 
derivation of e.g. Eq. (4). As explained in the quote above, the practical application 
of this simplifying methodology requires that we assume that the subsets of a sample 
explored by different particles can be viewed as random samples (‘disorder realiza- 
tions’) generated by some basic statistical properties of the global structure. In the 
context of stochastic geometry, this same assumption is more commonly called (spa- 
tial) stationarity, and it is a key assumption in many classical proofs in the field. In 
particular, the assumption of stationarity enabled the first proofs of the fundamental 
formulas of stereology. In a number of stochastic geometry results it is now known 
that a weaker ‘first order stationarity’ assumption is sufficient [1], and it might be 
interesting to see if the same is possible for the long-time limit results discussed here. 


4 Directional Measures and the Strong-Gradient Limit 


The concept of disordered media and statistical averaging can be particularly valuable to deal 
with the geometric complexity of biological tissues. We believe that further progress in the 
field can be achieved by merging microscopic geometric models, statistical [descriptions] 
of disordered media, and high-gradient features of the signal formation. 


Grebenkov (2016) 
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The gradient strength parameter set during a diffusion-weighted MRI experiment 
determines—roughly speaking—the scanner’s sensitivity to diffusive motion. The 
mobility of the water molecules in the sample affects the decay rate of the diffusion- 
weighted signal, cf. Sect. 1, and stronger gradients make this effect correspondingly 
stronger. Although stronger gradients thus produce weaker, harder to detect signals, 
these signals contain information about interesting features of the diffusion that 
cannot be observed at lower gradient strengths. The results for the short-time and 
long-time limits described in the previous subsections are mainly used at low to 
moderate gradient strength acquisitions. 

When we increase the gradient strength in a diffusion-weighted MRI experiment 
we first notice that the anisotropy, i.e., the orientation-dependence, becomes more 
significant. While we omitted this before, anisotropy is already a factor at the lower 
gradient strength experiments used for the concepts discussed in Sects. 2 and 3. In the 
general case, the long-time limit Dæ of D(t) considered in Eq. (3) is, for example, in 
fact a tensorial quantity related to the mean squared molecular displacements along 
different orientations. This anisotropy in the diffusion reflects the anisotropy in the 
sample’s micro-structure, so for example in the brain it is predominantly determined 
by the orientation distribution of neurites, cf. Sect. 1. At higher gradient strengths, 
the orientation-dependence of the diffusion can no longer be represented by a simple 
tensor, and a more complete set of orientational features becomes accessible. 

The neurite orientations are generally characterized by a so-called fiber orientation 
distribution function (ODF), which specifies the likelihood that a neurite is locally 
tangent to a given orientation, and the estimation of this object from strong-gradient 
experiments is a common problem considered in diffusion-weighted MRI. A large 
number of techniques have been proposed to deal with this question [14, 31, 32], 
but we will not discuss them in detail here. Instead we will point out that the ODF 
also has a stochastic geometrical analogue: the rose of directions [29]. The rose of 
directions is defined for fiber processes—a well-defined (locally finite) collection 
of randomly placed fibers. The rose of directions is the distribution of the direction 
tangent to a typical point on a fiber, where the meaning of a ‘typical point’ can 
be made explicit using Palm distribution theory. A stationary fiber process is an 
example of a stochastic geometrical model in the spirit of the work by Novikov 
et al. [21] that could be useful to model biological tissues. Furthermore, the rose 
of directions has a dual—the rose of intersections. These two objects are related 
through the Funk—Radon integral transform [9], which also appears naturally in for 
example the diffusion-weighted signal expression for narrow cylinders (the ‘fiber 
ball’ ODF) [14]. 

At even stronger gradients we move to another limiting regime—the strong- 
gradient limit, or localization regime [11]. Here we can still make use of the intuition 
developed in Sect. 2: any structure in the sample will act as a barrier for particles in 
its vicinity, and thus slow down the average motion in that region. Slower diffusion 
in turn implies less signal decay, and so the main contributions to the signal now 
come from particles localized near the barriers [28]. The localization effect carries 
non-trivial consequences for the diffusion time-dependence of the signal, but for this 
chapter we are more interested in the orientation dependence in this regime. 
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A simple way to understand the impact of the geometry’s anisotropy on the signal 
in the strong-gradient limit, is to look at a diffusion propagator with a compact support 
Q. At strong gradients the integrand in Eq. (1) oscillates rapidly, and as a result the 
integral indeed tends to 0. However, the rate at which it decays is in fact governed by 
the support Q, which is of course determined by the geometry. We can consider as 
an illustration the ‘indicator’ propagator which is 1 everywhere inside a sufficiently 
smooth and convex &2, and 0 elsewhere. The signal decay is then exponential with an 
exponent sup,..¢ gr [12], where the exponent, viewed as a function in q, determines 
Q completely. We could for example use this observation to recover the shape of a 
convex pore enveloping a diffusing substance, provided that the pore is much smaller 
than the typical diffusion length vDo t [6], but this approach is not limited to closed 
pores. 

The function q +> sup,¢g qr is called the support function of Q in stochastic 
geometry, where it can be associated with the rose of directions of a fiber pro- 
cess [29]. A similar interpretation can be given to the support function as it occurs 
here, leading to the definition of the barrier ODF [7, 8]. It must be mentioned that 
as these developments are very recent, their practical utility is currently still being 
investigated. 


5 Perspectives 


We have given three examples of concepts in diffusion-weighted MRI that also 
occur in stochastic geometry: the surface-to-volume ratio, which is related to the 
specific surface area commonly estimated in stereology; statistical homogeneity, 
which is related to stationarity; and the fiber orientation distribution function, whose 
stochastic geometry analogue is the rose of directions. In the context of neuroimaging, 
the common thread between them appears to be the concept of stationary fiber and 
surface processes from stochastic geometry, which are completely characterized by 
the specific length/area and the rose of directions. 

We believe further investigations in this direction could lead to new and useful 
methods for the analysis of diffusion-weighted MRI data, as well as novel research 
problems in stochastic geometry. In particular, we hope that the link to the mature 
statistical theory found in stochastic geometry may offer practical tools for the anal- 
ysis of uncertainty and variation in applications of diffusion weighted MRI: How 
much faith can we put in estimated anatomical quantities? Which quantities can we 
expect to be able to derive from given data? What are the conditions we should put on 
our data acquisition in order to be able to draw sound conclusions for the hypotheses 
that we ultimately hope to investigate? 
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Abstract Porous or biological materials comprise a multitude of micro-domains 
containing water. Diffusion-weighted magnetic resonance measurements are sensi- 
tive to the anisotropy of the thermal motion of such water. This anisotropy can be 
due to the domain shape, as well as the (lack of) dispersion in their orientations. 
Averaging over measurements that span all orientations is a trick to suppress the 
latter, thereby untangling it from the influence of the domains’ anisotropy on the 
signal. Here, we consider domains whose anisotropy is modeled as being the result 
of a Hookean (spring) force, which has the advantage of having a Gaussian diffusion 
propagator while still confining the spatial range for the diffusing particles. In fact, 
this confinement model is the effective model of restricted diffusion when diffusion is 
encoded via gradients of long durations, making the model relevant to a broad range 
of studies aiming to characterize porous media with microscopic subdomains. In this 
study, analytical expressions for the powder-averaged signal under this assumption 
are given for so-called single and double diffusion encoding schemes, which sensi- 
tize the MR signal to the diffusive displacement of particles in, respectively, one or 
two consecutive time intervals. The signal for one-dimensional diffusion is shown 
to exhibit power-law dependence on the gradient strength while its coefficient bears 
signatures of restricted diffusion. 
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1 Introduction 


Magnetic resonance has proved to be an extremely effective tool to peer into materials 
and tissues noninvasively. It manipulates the magnetic orientation of molecules per- 
vading the material into a natural precession which emits radio waves. By imposing 
a spatially-varying magnetic field (hence frequency of precession), the emitted radio 
frequency signal is made to encode in its spectrum the coordinates of the molecules 
that emit it. This way, the signal can be spectrally decomposed to trace back what pro- 
portion of it originates from where, that is, from which ‘voxel.’ Commonly achieved 
voxel sizes are in the neighborhood of a millimeter. 

Another use of a spatially-varying precession rate involves sensitizing the signal 
to the motion of the molecules; diffusion in particular. When the molecules trace 
out random (Brownian) paths where different locations they visit impart different 
precessional angles on them, their precessions lose coherence, attenuating the sum of 
their emitted radio waves. This attenuation of MR signal, specifically its response to 
the direction of the gradient in precession rate, can reveal or quantify how mobile the 
molecules are along different directions. The difference of mobility can arise from 
pore boundaries, impurities, cell membranes, etc. Hence the diffusion-attenuated 
signal encodes the influence of structures that can be significantly smaller than voxel 
dimensions. Tracing out axon bundles in human brain white matter is for instance a 
widely employed application of this principle. This modality of magnetic resonance 
imaging, which we refer to as diffusion MR, is the subject of this contribution. 

While the anisotropy (i.e., variance under a rotation transformation) of a single 
pore or a cell may be easily visualized, one would be mistaken to make a one-to- 
one connection with that and the anisotropy of the signal (i.e., the response of the 
signal to orientations of the specimen or the apparatus). For instance, the signal of a 
voxel consisting of an unaligned mixture of cylindrical aqueous compartments will 
be less sensitive to rotations than one consisting of an aligned bundle of cylinders. 
The anisotropy of the individual compartments is common in the two examples, but 
the aligned case has more ensemble anisotropy. 

In some sense, then, ensemble anisotropy confounds compartment anisotropy 
at the signal level, and eliminating it pronounces features at the subvoxel level. 
One way to achieve this is to take an average of the signal over all orientations. If 
the material allows it, it can be ground into a powder to that effect; hence the term 
powder averaging. However this is generally impossible in bio-medical applications. 
Then, repeated applications of a measurement protocol in different orientations is 
the avenue to follow. 

In this contribution, we are concerned with two particular diffusion MR schemes, 
single and double diffusion encoding (SDE and DDB), orientationally averaged in 
the aforementioned fashion to eliminate ensemble anisotropy. The single encoding 
scheme [44] is the bread and butter of most applications, employing a magnetic 
field gradient that remains on for a specified duration in one direction, and then 
the opposite direction after a specified delay (Fig. 1). The signal then encodes the 
probability of Brownian displacement between the application of the two pulses. 
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Fig. 1 The pulse sequence for single diffusion encoding, with time running along the horizontal 
direction 


Fig. 2 The pulse sequence for double diffusion encoding 


SDE has been the encoding scheme employed since some of the earliest studies 
of the powder averaged signal [5]. However, very recent theoretical [13] as well 
as experimental [1] studies have considered the orientationally-averaged signal for 
more general gradient waveforms. 

The double diffusion encoding (DDE) scheme employs two single-encoding 
blocks in succession [9], in different directions in general, before the signal is read 
(Fig. 2). This method has been studied extensively in recent years mostly because it 
allows the anisotropy of the microdomains to be quantified [8], which has important 
implications for medical imaging. The reader is referred to the reviews [4, 11, 27, 
40, 42] for in-depth presentation of the method. In a nutshell, the DDE technique 
encodes into the signal the joint probability of two Brownian displacements taking 
place between the two pulses of each block. For freely diffusing molecules, the two 
displacements are uncorrelated [36]. However, when restrictions, and arguably inho- 
mogeneities and forces, are present, this is no longer the case [24], hence imparting 
signatures of compartment size onto the signal. As mentioned above, DDE employ- 
ing gradient blocks in different directions is sensitive to anisotropy of subdomains 
inside the voxel [6, 8], but not independently of ensemble anisotropy [29]. 

While we do not consider restricted diffusion in the strict sense of restriction by 
hard walls in this article, we do respect the finite range of motion of the molecules by 
the aid of a harmonic attractive force [7, 20, 25, 46, 49]. This is an effective model 

which can mimic restrictive walls with reasonably tractable mathematics, and is 
actually approached when the gradient pulses are long [34]. 
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This chapter is organized as follows. We first present the signal arising from an 
effectively confined domain under double diffusion encoding. Afterwards we derive 
analytical expressions for its powder average and consider the “stick” geometry, 
wherein diffusion is fully confined in two dimensions. We then treat the case of single 
diffusion encoding as an extreme case of double encoding and give its corresponding 
powder average expressions. The special cases of “stick” and “pancake” geometry 
are compared to their counterparts for free diffusion. The article is concluded after 
discussions based on the results of the previous sections. 


2 Double Diffusion Encoding at the Compartment Level 


Here we derive the double-encoded diffusion MR signal arising from a compartment 
which is characterized by an effective spring force attracting the molecules toward 
the center. Such a force mimics the confining effect of walls, membranes, etc., with 
minimal mathematical burden. 

The spring (Hookean) force influences the motion of the molecules via a quadratic 
potential! 


V(x) = įx'Cx, (1) 


defining the confinement tensor C, which can be taken to have C7 = C without 
loss of generality. However, in order for the steady state molecule number density 
Ps (x) x e~”) to be normalizable, C must be positive (semi)definite.” Under this 
(or any) potential, the magnetization density p(x, t) evolves according to 


0, p(x, t) = (DV) -e "Ve" p(x, t) — ig(t) - xp, t). (2) 


Here, we have assumed that diffusion is governed by a spatially-uniform, possibly 
anisotropic diffusivity tensor D and that diffusion encoding is achieved by a magnetic 
field gradient waveform g(t). Note that we absorb the gyromagnetic ratio y into g(t) 
so that it has dimensions of time~! length~!.* 

The signal Se = f dx p(x, t) arising from a single such confined compartment 
under a general encoding waveform g(t) can be found thanks to the Brownian paths 
having a Gaussian probability measure under the potential (2) [49]: 


So = exp(—JaODO4O — Í ‘dr aca) f (3) 


ln units of the thermal energy scale kg T, where kg is the Boltzmann constant and T is the absolute 
temperature. 


? Vanishing confinement (i.e., free diffusion) has an unnormalizable steady state, but it can be 
handled. 


3 Alternatively, g(t) can be called a precession rate gradient waveform. 
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with the generalized encoding wave vector 


te 
q.(t) = f dre 2-9 g(r). (4) 
f 
Here, t is the duration of the encoding protocol, and 
Q = DC (5) 


is a matrix of equilibration rates. According to an equivalence between the signals of 
diffusion under a Hookean confinement and under restricting walls [34], a restriction 
of linear size L should have a confinement value of around C ~ 11/L?. For instance, 
the confinement value within walls 5 jum apart would be about C ~ 0.44 um~?. 
With water diffusing at D = 3 \1m?/ms, this would imply an equilibration rate of 
Q = 1.3 ms™!. In other words, it would take random walkers a few milliseconds to 
spread out roughly to their eventual distribution. 

As depicted in Fig. 2, double-diffusion encoding is achieved by a gradient wave- 
form consisting of two pairs of bipolar rectangular pulses, each with a given duration 
ô and separation A, and magnitudes g; and —g,; the minus sign is customary. The 
time between the leading edges of the second pulse of the first pair and the first pulse 
of the second pair, fm, is called the mixing time. The calculation of the signal (3) 
therefore entails very simple integrals, but in a cumbersome piecewise fashion. Upon 
significant simplification one finds 


Se(81, 82) = exp(—8] To) — 83 T.82 — 28, T x82) - (6) 
One may refer to the tensors 


T, = DQ? [0 — AA — ee — (1 — 07 ye + 295], a) 
2T,. = DQ 36 Q(tm dq —e 28)2 (1 _ e724)? ; (7b) 


respectively, as the self-coupling and cross-coupling tensors between encoding 
blocks.* For the orientational average below, we consider |g,| = |g>| = g, with a 
fixed angle ~ between them. 

As the free diffusion limit is approached, one can see that the cross-coupling of 
encoding blocks vanishes, as Tx ~ (D/2) 2657 A7(1 + Qô — Qim), due to displace- 
ments in separate time intervals being uncorrelated in pure Brownian motion [36]. 
The dependence on the mixing time fm does not enter until second order in confine- 
ment, in line with approximate calculations done for a spherical wall [29]. 


4T, is the same tensor that appears in the single-encoding signal S, = e78 "Tog [49]. In the free 
diffusion (2 — 0), it is easily shown that T, > D(A — ô /3), which recovers the free diffusion 
signal [43, 44]. . 
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3 Double Diffusion Encoding: Powder Average 


The orientational (powder) average of the compartment signal (6) is performed as 
follows: The vectors g, and g, form a plane with unit normal vector 


§1 X 82 


n= ‘ 
Ig X ol 


(8) 
One constructs the average by integrating over all orientations of (the plane normal 
to) ñ, and for each of these integrate over all orientations of the pair {g,, g3} within 


the plane, with their relative angle w fixed. The procedure hence described can be 
written as? 


s SE = Selga (B), ga +Y). (9) 


Here, g; (8) is a vector of magnitude g in the plane normal to 7, whose orientation 
is parameterized by the (in-plane) azimuthal angle 8, according to which we can 
identify g, > ga(8) and g, > g,(G +Y). 

We first take the in-plane integral 


dg =a (b) 


5a [< LAOR ACTE & I Le (10) 


which serves as the definitions for the intermediate quantities 5 (ñ) and aa(Q). 
According to Eq. (6), the latter is given by 


aal) = g (B)Togg(B) + 8408 + P)Toga(G +y) + 283 OTga + ¥)- 
(11) 


To anchor the angle coordinate 3, we define the in-plane cartesian coordinates u, v 
such that ĉ || g,(0) and Ŷ =n x ù, yielding 


g,(0) = ûg cos 8 + ŶÎg sin B. (12) 


‘i . : ‘ š * 5 7 5 ee 
After an exercise in trigonometric simplification, and denoting ù T,» = T, etc., 
one finds 


Fa(B) = Sa + pa cos 2p — oh sin 2G, (13) 


5We suppress the obvious limits of these integrals. 
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with 
Sa o o x x 
g2 = (Toa zz T?) + (T T TX) cos y ’ (14a) 
â 1 2 i . 
= = (Tp, — T?) Lreosty + Te, sin2v + (TX — T% ) cos + 2T% sin, 
(14b) 
On o o sin 2y o x x > x 
g2 = (Tou ~ To) 72 ~~ Tl + cos 24) + (Tà ~~ T>) sin Y% E 2T cos Y $ 
(14c) 
These depend on ñ through @ and Ŷ, of course. Hence via Eq. (10), 
S(a) = ay dB o cos 23+, sin 28 
2T 
= ool p;, + a) ; (15) 


where an integral representation of the modified Bessel function of order 0 was 
recognized [10]. The argument of the square root can be found after a semi-tedious 
calculation as 


pa + of = g [(T2, — Te) cosy + (TX — TX) + 4g*(Te, cos + TX), (16) 


and we rewrite the powder-averaged double-encoding signal (9) via Eqs. (10) and 


(15) as 
Z da _. 
S= f Fesn( V+). (17) 


For the actual evaluation of the integral, explicit expressions in terms of the polar 
and azimuthal angles (0, y) of #@ need to be substituted,° which are steps we omit 
here. Eyeballing how entries of T, and Tx appear in Eq. (17) via Eqs. (14a) and (16), 
one notes that it is useful to define the intermediate tensors 


M = g°(T. cosy + Tx) (18a) 
M = g°(T. + Tx cosy). (18b) 


Referring to their eigenvalues, in any preferred order (but the same for both), as m; 
and m;, and using the following shorthand 


SA = (sin 0 cosy, sin 8 sin y, cos 0), û = (cos 0 cosy, cos@siny,—sin@), and = (—sing, 


cos y, 0). 
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m;; =m —m;, (19a) 
fitz, = rt; — ñij, (19b) 
m} = my +j, (19c) 


the powder-averaged signal (17) attains the form 


S = emi f aes 0 dy e7 (mtp sin?) sin? (20) 
T 


x Io (Vom + 2m), [mnz + (miz + m33) sin’y] sin?@ + (m3 +m), sin’y)” sinto ) i 


upon substantial algebraic manipulation. 
Reference [13] found series expansions for this form in a different context. We 
can use its results. Namely, 


(oe) 


k 
eis 2 mk Ymk , where (2 1 a) 


BO ä x n—l 
S e D” (mz) (an) 2m + r) (n +k)! 
Ymk = > I(n—1)!  220m+D+1 m+l /(n+k+ 5)! lay 
(21b) 
(m3, k/2 (mp) he (mh) 


km)! & ij (mgtma 


qmk = 2Fi(m k, —2j;m+1— 2j; rma) , 
(21c) 


with the following three alternatives for Eq. (21b): 


2m\ > (ñp) (k +n)! iz 
yO = vT (ir) ee pnl + 5,—n;m+1; za), (22a) 
31 


mk — 22m+1 n!(k +n + 5)! 
(mz) "(k zi niet) 


n+m 


y? = 
Y nk ~~ 92m+1 aie 22ny! (k+n + 5)! 


1Fi(k+n+1;k+n + 3; ñp), 


(22b) 

G) Vm (2m\ S (ñp) k+n)! 
Vink = 22m+1 m > sae a m+1, k+n+3; mz) b 
(22c) 


Here, ; F;(. ..) are the confluent hypergeometric functions [3], tilde denoting regu- 
larization. Note that these expressions apply to the most general case, in which the 
confinement tensor (and thus the tensors that are functions of it) have three distinct 
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eigenvalues. Special cases associated with coinciding eigenvalues are discussed in 
the appropriate occasion later. 

Which alternative among Eq. (22) yields better convergence is a matter of how the 
eigenvalues m; are chosen to be ordered, by way of the sizes and signs of Mij- Asa 
general guideline it would be wise to order the eigenvalues so as to avoid a sequence 
that alternates in sign, and with a large expansion parameter. Take Eq. (22b) for 
instance. Given that ı F,(k+/+1; k+l+3; M3) > 0 and increasing for all my3, it 
would be beneficial to make mj, negative (and large if possible), while keeping m5, 
positive (and small if possible). For a given set of eigenvalues m;, ordering them 
in the fashion m, < m2 < m3 would be along this guideline, whereas the ordering 
Mm < m3 < Mı would result in a sequence with larger terms and alternating sign. 

Note however that while T, is a monotonic (decreasing) function of Q, T, is 
not; see Eq. (7). Through Eq. (18) this means that their mixtures M and M are not 
necessarily monotonic in the confinement @. Hence what ordering of the confinement 
eigenvalues (2; achieves what ordering in the eigenvalues m; and m; is a question 
which has an answer only on a case-by-case basis. Furthermore, the ordering of Q; 
that yields a desirable ordering of m; for a particular one of Eq. (22) may not produce 
an ordering of m; as desirable for the convergence of qmg in Eq. (21c). 


3.1 Axisymmetric Confinement 


We refer to the condition when two of the eigenvalues Q; of the confinement tensor 
coincide as axisymmetric confinement. Under this condition, the series expansions 
above undergo simplifications. 

The most drastic simplification occurs when Qı = Q2. That is, given that two 
eigenvalues coincide, assigning first and second place to them is the wisest choice 
as far as the evaluation of the series expansions (21) is concerned. 

First, the coefficient g,,. simplifies as! 


7For this, it needs to be noted [13, supplementary information] that the way qm arises in the 
calculation—before ever arriving at Eq. (21c)—is that it is the coefficient in the (double) series 
expansion of the Bessel function in Eq. (20): 


co Uk 
IoC...) = D D qmk sin?” p sin%ð. 


Upon all coefficients in the argument except for m13 vanishing due to axisymmetry, one has 


Io(m; sin’) = 5 (mi) sin**9 
13 a , 
= (k!)222k 


which, comparing to the previous (double) expansion, implies Eq. (23). 
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Smo (73) 
3— | evenk 
dmi = { O'O (23) 
0, odd k 
The coefficient Ymg, on the other hand, loses the interior summation in Eq. (21b) 
due to mj, vanishing. The remaining summation can be identified according to the 
definition of hypergeometric functions [3] as 


ST (2m\_ k! ase 
Ym = seat, ep Ty e+ A (24) 


which yields via Eq. (23) and Eq. (21a) 


: singe fD R)” 1Fi(2n + 1; 2n + 3; my, 


This expansion is not so sensitive to the ordering of eigenvalues number | and 
3, as the expansion parameter (7m}3) is squared, and the hypergeometric function 
1Fı(2n+1; 2n+3/2; mi3) > 0 for all arguments. 

We depict in Fig. 3 the evaluation of the powder averaged signal (25) for axisym- 
metric confinement for representative values of encoding parameters. In Fig. 3a, the 
confinement is anisotropic (prolate), whereas in Fig. 3b, it is (nearly) isotropic. The 
bell-shaped dependence on the relative angle 7 between the gradient directions is 
seen in both cases, which is a sign that diffusion is not free [29]. This dependence 
is due mainly to the exponential prefactor in Eq. (25) that has nothing to do with 
the difference between the confinement eigenvalues (anisotropy). When the mixing 
time is increased, the bell-shaped modulation, indicating confinement regardless of 
anisotropy , stops overwhelming the relatively smaller influence of the rest of the 
expression (25): see Fig. 3a where the confinement is anisotropic. In an isotropic con- 
finement, on the other hand, angular modulation simply disappears when the mixing 
time is increased (Fig. 3b), illustrating that the angular modulation that survives the 
increase in mixing time is due only to compartmental anisotropy, and not due to the 
fact of confinement (or to ensemble anisotropy, which was already eliminated by 
powder averaging). 


3.2 Insights from Two Dimensions 


The signal expressions for double encoding are a bit unwieldy to get a conceptual 
handle on. However, some insight can be gleaned from considering the orientational 
averaging in two dimensions. 

Obviously, in the spirit of Eq. (9), the 2D orientationally averaged signal can be 
written as 
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qa 


(c) 


Fig. 3 Dependence of powder averaged signal (25) on the angle ~ between double-encoding gra- 
dient directions. a Prolate confinement &, with eigenvalues {1, 1, 0.1} in arbitrary units. A = 10 
and ô = 1, in units of 2r". The mixing time fm was set to the following multiples {1, 2, 6, 30} of 
ô. The gradient strength is set such that Dog?/ of = 0.16, with Do being the unit of diffusivity. 
b Isotropic confinement Q, with eigenvalues {1, 1, 1} in arbitrary units. A = 3 and 6 = 2/3, in 
units of V . The mixing time fm was set to the following multiples {1, 3/2, 3, 6} of ô. The gradient 


strength is set such that Dog”/ Q3 = 1. c Free diffusion with varying anisotropy (prolate) for com- 
parison, with diffusivity D eigenvalues {Do, eDo, eDo}. Across the three groups of plots, diffusion 
weighting b = g*67(A — 6/3) takes the values {0.4, 1.4, 10} in units of De , whereas each group 
of plots have the eccentricity parameter set to {0.01, 0.1, 0.2} 


= d 
Sop = J 2 Se(e (8), g6 + Y)). (26) 


That is, there is no normal vector to integrate over; everything takes place in an (x, y) 
plane. The same steps Eq. (10) through Eq. (15) apply, and one has 


Sop _ oP TH(T.+1. cos v) Io(g?(TP = T3) cos w ae (Ty = i )) . (27) 


The dependence on the relative angle Y% of the gradient directions occurs both in the 
exponential attenuation factor and in the Bessel function. The angular dependence 
in the Bessel function has to do with anisotropy (7? — T;) which the exponential 
factor is insensitive to due to the trace. In the exponent, on the other hand, the angular 
dependence is controlled by TrT,,, whose presence is due to confinement (since 
T, — 0 in the free diffusion limit). For large mixing times (tm2; > 1) the latter 
drops out and the angular modulation due to anisotropy is liberated. However, for 
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smaller mixing time, it turns out that the angular dependence of the exponential factor 
dominates, which is due to confinement, suppressing the signature of anisotropy. 

An interesting double-encoding scheme is its “symmetrized” version [35], which 
fixes Y = 7/2, but varies the magnitudes g; = g cos a and g2 = g sin Q as a function 
of some parameter œ. In that scenario, one can easily calculate that 


sym g2 2 
sy = aa an (AT2 + AT?) + (AT? — AT?) cos 4a) , (28) 


showing that the modulation due to confinement in the exponential factor drops out, 
and one of pure anisotropy remains (with AT, denoting TP — T; etc.). The result 
is a ‘cleaner’ version of the signal modulation wherein the confinement anisotropy 
is the only source of angular modulation characterized by the cos 4a dependence. 
It should be remembered though that the confinement model is purely Gaussian, as 
such does not account for compartmental kurtosis. When truly restricted diffusion is 
considered, the compartment anisotropy and compartmental kurtosis both yield the 
same type of angular dependence compromising the interpretation of such angular 
dependence except when the compartments are isotropic [35]. 


3.3 One-Dimensional Diffusion Under High Gradient: g7? 
Scaling 


A special case of interest is when the compartment has an extremely elongated shape, 
resembling a “stick”. Such extremely anisotropic shapes combined with an asymp- 
totically large gradient strength (g — oo) tend to exhibit power-law dependence on 
the gradient strength g or the wave vector q = ôg. In what follows, we confirm that 
the orientationally averaged signal (25) is no exception. Physical interpretation of 
the limits or extremes involved is remarked on at the end. 

The two transverse confinement eigenvalues Qy = Q2 = Q1 approach infinity 
while Q3 = Qy is finite. Accordingly, mı = m2 = m, = 0 and m3 = my, is finite 
(similarly for m;). Combined with a gradient g — oo, noting that mj ~ g? and 
my ~ g? due to the definition (18), the orientationally averaged signal (25) assumes 
the asymptotic form 


-1 


z p my \” 
Sstick ~ | 2my,/ 1 — (| — ; (29) 
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see footnote 8 for details.* When we substitute from Eq. (18), the power-law depen- 
dence (~ g’) becomes obvious: 


1 


Ssick ~ | 28° (T? + Tř cos) |1 ( (30) 


This result should be compatible with the findings of Ref. [13] in the free diffusion 
limit (Q — 0). It is easy to see that Eq. (7) implies TĚ — 0 and T; => D(A — 
5/3) as 2 — 0, yielding 

Sik ~ (288D (A — $) sind) G1) 


D\ 00 
which is in fact the powder averaged signal of a rank-1 diffusivity tensor ( 0 0 0) 


0 00 
F 12 5 1+cos wy 0 0 
under the rank-2 measurement matrix g°d (A — $) 0 1—cosy0 | of DDE, as 
0 0 o0 


predicted by Ref. [13]. This scaling (~ g~*) of the signal of stick-compartments 
under rank-2 measurements (planar encoding) has been recently confirmed in vivo 
[1]. Note that the expression is invalid for y = 0 and Y = m where the two encoding 
directions coincide, and the corresponding measurement matrix is rank-1. 

Lastly, we address the physical interpretation of the limits of large confinement 
(Q) and gradient (g) values. Note that the final expressions such as Eqs. (21)—(22) 
and (25) come out in terms of the eigenvalues of the matrices M and M, which are 
in turn defined in Eqs. (18) and (7). The sense in which a confinement eigenvalue Q 
and gradient strength g is large follows in particular from Eq. (7): In the same order 
as the limits have been carried out, infinite (transverse) confinement (Q, — co) 
is to be interpreted physically as g*D, 2r <_ 1, while infinite gradient (g —> co) 


8At the extreme of mj = m2 = m] = 0, the signal (25) becomes 


Tan GS C) my?" Fi (2n + 1; 2n + 3; =) 
stic! = n jJ 22n+1 (2n+4 5)! 


The hypergeometric function can be rewritten by a so-called Kummer transformation as 


| F\(2n + 1; 2n + 3; fay) =e" | Fy (4; 2n + 35 ty) ~ (2n + iVa t, 


where the second step uses the asymptotic form (mm — oo) of the Hypergeometric function [10]. 
Hence, the signal attains the form 


2n om 
Sstick me n+l Dt > 


n=0 


converging to Eq. (29) for |m) /m)| < 1, which can be verified by carefully considering the func- 


tional dependences arising from the explicit form (18) of the matrices M and M (with the exception 
of the encoding angles 7 = 0 and Y = m where the two encoding directions coincide). 
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is to be taken to mean DQ), > 1. One may write D Kg? K D'R in 
short. It should be noted that this order-of-magnitude treatment ignores potential 
exceptions or extremes which may arise from very particular combinations of the 
protocol parameters 6, A, tm, and w. 


4 Single Diffusion Encoding 


The powder-averaged single-encoding signal can be obtained by way of “hacking” 
its double-encoded counterpart. One notes that S = e~ "Tog is the single-encoding 
signal at the compartment level [49]. Then the features of the form (6) to get rid of 
are (i) the presence of a second vector g, unequal to the first, (ii) the presence of 
cross-coupling (Tx), and (iii) the double occurrence of self-coupling. The first is 
hacked away by setting 7 = 0. For the second, one simply sets Tx = 0.° The result 
is Sp = e778" Tes, suffering from the third problem above, which is fixed by replacing 
g > g/V2. All this yields via Eq. (18) 


M=}gT, =M, (32) 


which is the substitution that converts the powder-average expressions for double- 
encoding into those of single-encoding. 

With this condition applied, the m;;’s appearing in Eq. (21) turn into m;;’s. How- 
ever this alone does not produce diastc simplifications such as reducing summations. 
Rather than using Eq. (21), in fact, it is better to note that other results in Ref. [13] are 
quite suitable in the case of single-encoding. Since the compartment signal has the 
form Se = e~ ™(T-88"), with the matrix gg' axisymmetric by dint of being rank-1, 
the results of Ref. [13] for axisymmetric diffusion or measurement tensor can be 
applied. Written in terms of the parameters of the present discussion, the relevant 
formulas of Ref. [13] indicate the following alternative expressions 


Sa) ee T? a (= + 14. TT 
S a Py iS va aFi(4,- n; l; TTE ) > (33a) 
n=0 3 
2 oo g” (T? — i 
(2) _ a-g T? ` ` 
SY =e aa sume iF \(n + 1; n+3:g (Te — TO). (33b) 


n 


o0 g” (T; a oe 


59 ee a ee 


n=0 


2F2($,n +1; 1 n+ 3; g’ (T - T's 


(33c) 


°Physically, this can be imagined as the limit t,, > 00. However, physics is not necessary. We 
simply have a set of expressions, e.g. Eq. (21), containing T, and Tx via Eq. (18) via Eq. (19), and 
we want to remove instances of T}. 
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F(2n+1; 2n+3; £(2T?-Tf-T?)), 


n n ° 9\2n 

54 — Vt eR (rs (Tr — T3) 
2 n=0 16" (2n+3)! 
(33d) 


for fully anisotropic & (hence T,); the axisymmetric case is taken up below. These 
summations are subject to the same guidelines that followed Eq. (22) except that the 
caveats about conflicting eigenvalue ordering do not apply here. The single-encoding 
expressions here only involve the self-coupling tensor T, which is a monotonically 
decreasing function of &2; see Eq. (7). Therefore the ordering of the eigenvalues T,° 
and Q; are certain to be in exactly the opposite sense of each other. 


4.1 Axisymmetry and the Power-Laws for Confined diffusion 


The first alternative in Eq. (33) needs special care when Tp = T} and the argu- 
ment of the hypergeometric function > F}(...) diverges. Invoking a property of the 
hypergeometric function,'? and renaming T? = T? and T? = Tẹ, one finds! 


5. = VT ee T36.4) erf(¢,/ Tp (6, A) — TP, Ay) 
axy = ——e Ae 
a gy Tp (6, A) — T? (Ô, A) 


(34) 


Here, we have also explicitly denoted the dependence on the encoding protocol’s 
timing parameters as per Eq. (7). Note that in the free diffusion limit (2 — 0, see 
footnote 4), the known single-encoding powder-average signal [2, 18, 48] 


(m tn— 5)!m! 
(m — 4)!(m+n)!- 


Tim x"2Fi(m+3, =n; m+1; 3) = (-y)" 


11 Alternatively to using the results of Ref. [13] here, one may go back to the integral expression 
(20). First, we note that axisymmetry, with the choice Q1 = (22, makes the integrand independent 
of ọ, yielding 


al 
ll 
Nie 


on. a i 
e7 a [ doos oes" 1o(mis sin?) : 


We are not aware of a closed-form evaluation of this integral. However, its special case (32) relevant 
here, making the arguments of the exponential and Bessel function match, has the result [12, 6.625— 


4] 
es erf (\/2m:,) F enle ETE) 


2 /2m3, 2 g/T3 -TP 
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ee Gee erf (q (A — i= D.)) 
i qy (A - §)(Dy— Di) 


with q = gô, is recovered. 
Two special limiting cases follow. 


, (35) 


4.1.1 Stick: § « g7! Scaling 


When particles have negligible latitude to move in the transverse direction, the 
orientationally-averaged signal (34) assumes a special form. In terms of the con- 
finement model, this corresponds to 2; — oo, which implies T? — 0 via Eq. D 
The signal (34) then becomes 


Sani ef(eVTED) eo VR Vk 
2 8 7/0, A) 28y TROA 24, (A = 2)D, 
(36) 


The large gradient regime has been important in identifying stick-like compartments 
via the q7! scaling, which has been observed in white-matter areas of the brain and 
has been interpreted with the assumption of a free one-dimensional diffusion [23], 
which is adequate for channels of straight long channels of infinitesimal diameter. 
A notable exception is Ref. [33], which has incorporated the effects of finite size 
and curvature to offer an explanation for any deviation from the g~! scaling, most 
apparent within gray matter. Ref. [33] also pointed out that such a scaling is not the 
true asymptotic behavior of the signal; the latter is rather dictated by the Debye-Porod 
law yielding q~* scaling for narrow pulses [38], while an even steeper attenuation is 
predicted when the pulses are wide. 

The above expression suggests that a similar decay is expected for the gradient 
magnitude rather than the q-value. The crucial difference is in the dependence on 
the timing parameters {6, A} of the SDE sequence, see Fig. 1. It would thus be 
interesting, e.g. in white-matter, to investigate whether the dependence on the timing 
parameters is more like 1/,/A — 6/3 (free difusion along the fiber) or 1/\/ TẸ (6, A) 
(confined diffusion along the fiber), which would inform about the diffusion process 
along the axons. 


12The limits Q, — oo and g —> œ are to be interpreted in the same sense as Sect. 3.3. 
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4.1.2 Pancake: S œ g~? Scaling 
For completeness, we consider the opposite case where particles are able to spread in 


a plane, but not along the normal. Here, 2; — oo, implying T} — 0 via Eq. QP 
One then has!4 


VT 8TA) erfi(g T? (ð, Ay) goo 1 21-0 1 
gT? ©, A) 28°T? (ô, A) 24?(A — È) Dı ` 
(37) 


Spancake = 


Thus, the orientationally-averaged signal attenuates at a faster rate than in the case 
of sticks. Similarly though, the dependence on the timing parameters are different in 
free and confined diffusion scenarios. 


5 Discussion 


We have provided, for the first time, explicit expressions for the orientationally- 
averaged SDE and DDE MR signal intensity for structures represented by confine- 
ment tensors [49]. The latter is the effective model of restricted diffusion when pulses 
are long enough for the diffusing particles to traverse distances larger than the pore 
size [34]. As such, our findings are relevant for a broad range of porous materials 
featuring isolated, small pores. 

The counterpart of these results for compartments of free anisotropic diffusion 
were given in Ref. [13] for arbitrary encoding waveforms. Reference [13] showed 
inter alia that the absence of axisymmetry could be discerned from the decay of the 
orientationally-averaged signal by employing the most general formulas. Though not 
rigorously studied here, we expect such arguments to be valid in the case of confined 
geometries as well by employing Eqs. (21)—(22) and (33). 

For the time being, considering arbitrary waveforms for confined compartments 
seems extremely challenging. However, taking confinement into account is impor- 
tant, since the free diffusion model lacks features (such as the dependence on relative 
angle in double encoding) that a realistic signal will bear, see Fig. 3. Such bell-shaped 
angular modulation was related to the radius of gyration of the pores [24] and has 
since been used to estimate the apparent size of pores [4, 17, 29, 30, 41] via DDE 
measurements. The underlying reason was thought to be a restriction effect [11] that 
not only leads to an anisotropy of the diffusion process at a length scale smaller 
than the pore size [26, 29, 31, 32], but also makes the apparent diffusion coefficient 
depend on the diffusion time [15]. Thus, this effect is absent when free diffusion is 
thought to take place within individual pores, see Fig. 3c. Our results indicate that 


13The limits Qı — œ and g — ov are to be interpreted in a sense analogous to Sect. 3.3, with L 
and || interchanged. 


14The imaginary error function has the properties i erfi(z) = erf (iz), and erfi(z) ~ e?z y/T. 
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the confinement tensor framework is capable of capturing such angular modulation, 
which makes it a suitable representation of diffusion within microdomains [22, 34, 
49]. This is important in applications like q-space trajectory imaging [47] and dif- 
fusion tensor distribution imaging [45], which aim to characterize the structure of 
subdomains using general gradient waveforms. 

Another angular modulation that is apparent in Fig. 3 is w-shaped, which was 
pointed out by Mitra [24] for randomly distributed sticks. Later, it was proved that 
the dominant contribution to such angular modulation had the functional form cos 27 
for fully restricted structures and cylinders of finite diameter [28]. Such modulation 
that manifests itself at twice the “angular frequency” [19, 24, 28] is present even when 
diffusion within the subdomains is envisioned to be free as long as it is anisotropic, see 
Fig. 3c. Thus, such modulation is truly indicative of the anisotropy of the subdomains, 
be it free (Fig. 3c), confined (Fig. 3a, b), or truly restricted [28, 32]. For more 
information on such anisotropy, the reader is referred to the review on this topic by 
Tanus et al. [14] in this book series. 

From a mathematical point of view, the expressions in Ref. [13] provided the 
Laplace transform of a tensor distribution, which includes rotated copies of a given 
diffusion tensor wherein all orientations are equally likely, thus extending the mod- 
eling approach that employs parametric diffusion tensor distributions [16, 21, 37, 
39] to a new type of tensor distribution. Evaluating the signal, in a similar fashion, 
for confinement rather than diffusion tensors can be regarded as the evaluation of a 
transform whose kernel is the compartmental signal given in (6) instead of the kernel 
of the matrix Laplace transform e~®?. 


6 Conclusion 


We have given analytical expressions for the orientationally averaged diffusion MR 
signal originating from confined anisotropic compartments for two relatively simple 
encoding schemes. A number of observations related to signal modulation and power- 
law tails were made for such confined pores. These findings complement and extend 
the exact expressions for locally free diffusion provided in Ref. [13] to confined 
diffusion albeit for SDE and DDE measurements. 
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Abstract Clinical tractography is a challenging problem in diffusion tensor imaging 
(DTI) due to persistent validation issues. Geodesic tractography, based on a shortest 
path principle, is conceptually appealing, but has not produced convincing results so 
far. A major weakness is its rigidity with respect to candidate tracts it is capable of 
producing given a pair of endpoints, showing a tendency to produce false positives 
(such as shortcuts) and false negatives (e.g. if a shortcut supplants the correct solu- 
tion). We propose a new geodesic paradigm that appears to overcome these problems, 
making a step towards semi-automatic clinical use. To this end we couple the DTI 
tensor field to a family of Riemannian metrics, governed by control parameters. In 
practice these parameters may allow for edits by an expert through manual selec- 
tion among multiple tract suggestions, or for bringing in a priori knowledge. In this 
paper, however, we consider an automatic, evidence-driven procedure to determine 
optimal controls and corresponding tentative tracts, and illustrate the role of edits to 
remediate erroneous defaults. 
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1 Introduction 


Tractography aims at reconstructing bundles of nerve fibers in the brain (aka tracts) 
from diffusion weighted magnetic resonance imaging (DWI), the only neuroimaging 
technique enabling non-invasive in vivo imaging of the brain’s fibrous structure. 
Unfortunately, persistent issues curb clinical progress [14, 53]. Decades after its 
inception [5, 7, 23, 30, 38, 47], lack of consensus and even skepsis as to its clinical 
feasibility prevail. This has sparked new incentives to specifically address current 
limitations in quantitative evaluation studies and international competitions [9, 12, 
13,.37,.39, 51]. 

The geometric rationale for Diffusion Tensor Imaging (DTD, in its original form 
proposed by O’Donnell et al. [43] and, from a somewhat different perspective, by 
Lenglet et al. [31], and subsequently adapted by Fuster et al. [21], Hao et al. [24, 25], 
and several others, stipulates that one can ‘geometrize away’ local diffusivity patterns 
inside the brain. The idea is to incorporate anisotropic diffusivity of water in brain 
white matter, viewed as a porous medium with an orientational preference along axon 
bundles [57], into the intrinsic geometry of a suitably defined, curved manifold (akin 
to the geometrization of gravitational forces in general relativity theory). A similar 
approach is taken by Aumentado-Armstrong et al. [4], where a curved manifold is 
used to model conductivity of electrical signals in the heart muscles. In the case 
of DTI a Riemannian manifold presents itself, since its defining metric represents 
a positive definite quadratic form (or inner product) that can be formally mapped 
one-to-one onto the DTI tensor. Due to its modest performance, however, geodesic 
tractography has been largely abandoned in return for other approaches. A notable 
exception is the probabilistic approach by Hauberg et al. [26] and Schober et al. [55]. 

Our goal is to provide a versatile deterministic Riemann-DTI geometric paradigm 
for DTI geodesic tractography, revisiting original ideas (loc. cit.), which overcomes 
some of the main weaknesses, such as producing false positives (e.g. shortcuts) and 
false negatives (e.g. a shortcut is obscured by the correct solution). Since the mapping 
of DTI data to biologically meaningful tracts is generally ill-posed, we aim for a 
flexible metric equipped with control parameters. This admits adaptation to fiducial 
“ground truth’ tracts, in which data-extrinsic knowledge may be incorporated (e.g. via 
manual edits by an expert or via machine learning). The parameters control a locally 
smooth, spatially varying so-called ‘3-bein’, or triad, detailed in Sect. 2 (cf. Savadjiev 
et al. [54] and Piuze et al. [48] for a similar idea in the context of myofiber geometry 
from DTI, based on Cartan’s method of moving frames). In Sect. 3 we perform 
experiments to illustrate the theory, provide proof of principle, and present results in 
the context of a simple DTI-tractography phantom for the sake of illustration. There 
are no obstacles for application on more sophisticated simulated or real data. 
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2 Theory 


Differential geometric approaches in DWI are not new [1, 3, 15, 20-22, 24, 25, 31- 
33, 43, 46, 48]. The premise underlying the Riemann-DTI paradigm is that tissue 
microstructure imparts non-random barriers to water diffusion [8, 57]. The fibrous 
nature of brain white matter, comprising bundles of elongated axons connecting 
nerve cells in surrounding grey matter regions, facilitates mobility of water molecules 
along fiber directions. Since DTI captures the main diffusion anisotropy, it is natural 
to stipulate a Riemannian metric proportional to the inverse of the diffusion tensor. 
In this way, a relatively large mean free path is tantamount to a relatively short 
Riemannian distance, so that the problem of tractography can be related to a geodesic 
(‘shortest path’) problem. Candidate tracts can then be obtained by direct integration 
of the geodesic equations, by functional minimization of the Riemannian length (or 
related cost) functional for curves with fixed endpoints, or (with some care) by 
inference from the Hamilton-Jacobi equation [28, 45, 50, 52]. 

Despite the appealing heuristics supporting the geometric paradigm, there are 
serious caveats we need to take into consideration: 


1. In a geodesically complete space any pair of points is connected by at least 
one geodesic, raising the issue of ‘false positives’ (curves not corresponding to 
meaningful tracts). 

2. A well-posed relation between geodesics as ‘paths of least resistance’ and mean- 
ingful neural tracts is not self-evident. Tissue microstructure, which remains unre- 
solved at scanner resolution, induces mesoscopic diffusivity patterns involving 
more complex factors than plain presence or absence of nerve axons [41, 42, 
44]. As a consequence, there may be many a priori equally viable microstructural 
explanations for any given DTI image,' so that a one-to-one mapping between 
DTI and Riemannian metric is unlikely to work for tractography. 


We will address both concerns and outline our strategy towards an improved frame- 
work. 

Ad caveat 1: Geodesic completeness , tantamount to (huge) redundancy, may be 
used to our advantage, provided two conditions are met: 


(i) Meaningful tracts correspond, to acceptable approximation and at least piece- 
wise, to geodesics. 

(ii) One can identify true (or reject false) positives, based on some deterministic or 
probabilistic criterion. 


The first condition is our main hypothesis, the second one is a constitutional part 
of the geodesic tractography problem. The problem then boils down to finding ‘the 
right’ metric together with effective connectivity criteria for pruning its geodesics 
[2, 50, 56]. Streamline tractography, in its simplest form a singular limit of geodesic 


'This fact also implies that ‘ground truth’ simulations must be interpreted with great care in order 
not to penalise experimental results that deviate from the stipulated ground truth used to create a 
phantom if such results are equally compatible with data evidence. 
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tractography based on a degenerate metric, lacks completeness, a direct consequence 
of its first order nature, which prohibits generic endpoint constraints. As a result, 
odds are that two fiducial endpoints (picked by an authoritative expert, say) fail to 
be connected, leaving us in a quandary how to repair for ‘false negatives’. Clearly, 
geodesic completeness requires (at least) second order schemes. 

Ad caveat 2: To allow for prior knowledge or retrospective corrections we do not 
determine a unique metric in terms of DTI data evidence a priori. Instead we aim 
for optimal control parameters for a family of metrics and induced geodesics in a 
joint (semi-)automated procedure. The premise is that, due to unknown microstruc- 
tural factors, apparent diffusivities reflect fiber orientations at best qualitatively. By 
investigating the parameter unfolding of the family of metrics along with geodesic 
pruning we may investigate whether a stable result compatible with ground truth can 
be achieved (bearing in mind footnote 1). To this end we have conducted a feasibility 
study on the Fibercup [13, 49] simulator to clarify all conjectured features of our 
approach in a simplified context. There are no fundamental obstructions for appli- 
cation to more sophisticated phantoms or to real data, but this elaboration is left for 
future work. 

For computational reasons one could employ any coordinate basis on the tan- 
gent bundle TM induced by an arbitrary coordinate map. Such a basis is com- 
monly denoted by {0;+0/0x'}j;=1,2,3. (A natural choice would be to employ the 
same coordinates as those given by a Cartesian coordinate frame of the associated 
Euclidean space.) However, given a suitable metric g;;, we may instead opt for a 
special, g-orthogonal (non-coordinate, or anholonomic) basis {é,}¢=1,2,3. This is the 
triad alluded to in Sect. 1. If we write the new basis vectors in terms of linear com- 
binations of the coordinate vectors,” 


ea = el ði , (1) 


then the coefficients e!, define the transformation matrix relating the general coordi- 
nate basis to the triad. We may define a new metric holor with entries hap relative to 
the triad by the standard change-of-basis formula: 


gijei,e] = han- (2) 


The triad can be chosen so as to put hap into a convenient form. In order to illustrate 
this ‘gauge fixing’, let us assume that the dual metric g®™ is identified with the DTI 
matrix D, as originally proposed. Take h=diag(1/4!, 1/7, 1/23), in which the A“ 
are the eigenvalues of D. In the parlance of classical matrix theory [27], ea is then 
the eigenvector of D associated with àf, which, in turn, defines a rank-one matrix 
Za =€q Q ea (‘Frobenius covariant’). Equation (2) is the geometrical counterpart 
of the classical Lagrange-Sylvester matrix decomposition, D= eee A“ Za, and the 
eigenvalues are our controls. We may vary the A“ and observe the effect on the 


The Einstein summation convention for identical upper/lower index pairs applies throughout in 
tensorial equations. 
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metric, induced geodesics, and ultimately fiber tracts. This parameter freedom unifies 
originally proposed models for the connection between DTI and Riemannian metric 
[21, 24, 25, 31, 43], since all instances can be obtained by slick (local or global) 
choice of 4“. This includes streamline tractography via singular perturbation theory. 

To find a geodesic we fix seed and target points, A and B say, and minimize the 
length functional for curves connecting these points. This can be extended to any pair 
of regions provided one employs an efficient algorithm. Solving the Hamilton-Jacobi 
equation (with the help of fast marching methods) seems attractive in this respect, 
but, by design, provides only global minimizers, and is therefore not likely to solve 
the shortcut problem. Instead we opt for direct, coarse-to-fine minimization of the 
parametrization invariant functional 


Lyly) = f SuaM OHO dt, @) 


in which 
y: (ta, tp] > R? : th x(t) (4) 


is an arbitrarily parametrized curve connecting A =x (t4) and B=x(tg) in R? and x 
denotes its derivative. We embed the metric in a multiresolution family, with scale 
parameter o € Rt, according to the multiplicative scheme proposed by Florack et al. 
for positive symmetric matrices [16, 18], viz. if g is the Gram matrix with entries 
Jij» then 

g(x, 0) = exp ($o *Ing)(x)) , (5) 


in which exp and In are matrix-exp and matrix-log, ¢, is the L'-normalised isotropic 
Gaussian kernel of scale o [17, 19, 29, 34], and x denotes entry-wise convolution. 
(This definition ensures that the dual of a blurred metric equals the blurred dual 
metric.) Since the metric can be shown to become Euclidean in the o —> oo limit under 
suitable, weak conditions, the asymptotic minimizer is the straight line connecting 
A and B. As one gradually decreases o, this minimizer (geodesic at resolution 1/0) 
is expected to deform likewise gradually, while adapting to the refined metric (at 
this level of rigor we ignore the problem of scale space bifurcations, cf. Damon [10, 
11]). A higher resolution geodesic can then be found by adding a few control points 
roughly equidistantly along the curve in proportion to resolution, and minimizing the 
multivariate function of the control points representing our discretization of Eq. (3). 
Now suppose we have a (local) minimizer at some scale ø, then along with a scale 
refinement o —> o — do (we take do «o) we increment the number of control points 
along the curve, and seek a new minimizer in the vicinity of the previous one. In 
this way one can arbitrarily refine the geodesic curve (until grid scale, if needed), cf. 
Fig. 1. 

The multiresolution scheme for geodesics sketched above applies to each fixed 
member of the (A!, A?, A7)-family of metrics, and should not be confused with opti- 
mization with respect to the unfolding of this family as a function of (A!, A7, 43). 
The details of this are given in the next section, notably Eq. (6). 
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Fig. 1 Coarse-to-fine minimization of the length functional in Eq. (3) is implemented as an iterative 
multivariate function minimization in which the number of variables gradually increases in propor- 
tion to resolution. The control points represent a Bezier curve (blue) which by default we initialize 
by the unique Euclidean geodesic (dotted line), although this initialising curve may be manually 
overruled. In this example there are two control points, Pz and P3 (recall that the endpoints, P; and 
P4 in this case, remain fixed). Note that the control points in general do not lie on the curve itself. 
These points are part of the ‘behind-the-scenes’ machinery in the minimization procedure and in 
themselves do not have any anatomical meaning. However, given k points in space, we may con- 
struct a Bezier curve through these points (although not uniquely). This may be useful for manual 
initialisation or correction by an expert, based on anatomical landmarks. 


3 Experiments 


In the following experiments we use an operational procedure to automatically select 
‘the right’ (anisotropic) scaling of the control parameters 4° for the triad, relative to 
the original eigenvalues of the diffusion tensor. To begin with, we have opted for a 
single, global parameter £ € (0, 1], introduced so as to rescale the diagonal metric h?” 
(the dual of h in Eq. (2)) by? diag(1, £, €). The global character of ¢ allows us to leave 
à! unscaled, since any scaling of à! can be absorbed into £. Effectively this yields 
an €-parametrized family of Riemannian metrics g,, replacing the unscaled metric 
9=Qe=1 in Eqs. (2), (3) and (5). However, we only use this scaling whenever the 
fractional anisotropy (FA) [6] of the diffusion tensor is large enough (here we choose 
FA =0.15 as the ad hoc threshold) to ensure a well-defined main eigendirection.* 
Below this threshold the diffusion tensor is kept unscaled. As a result we obtain an 
€-parametrized family of geodesics ys, cf. Eq. (4). These geodesics are computed one 


3In principle, the control parameter triple (A!, 47, 43) permits local adaptation leading to a 3- 
parameter family in every voxel. The singular limit € =0 corresponds to streamline tractography and 
is excluded for its geodesic incompleteness. Values ¢ > 1 may affect the ordering of the eigenvalues 
of h™. 

4This ad hoc anisotropy threshold calls for a more rigorously motivated alternative, but it serves 
our purpose in this feasibility study, viz. to ensure well-posedness. 
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by one by the coarse-to-fine scheme outlined at the end of Sect. 2 for each fiducial 
pair of endpoints and each setting of €. 

Subsequently each geodesic thus obtained is quantitatively evaluated in terms of 
the tract length unbiased, nonlinear connectivity functional [2, 50, 56] 


Lie) 
Lg (Ye)? 


Ce) = (6) 


in which 7 denotes the standard Euclidean metric (so that L, (y) is the usual length of 
y), recall Eq. (3). Note that in the denominator we use the unscaled diffusion metric 
g = ge=1, for which the curve ys is, in general, not a geodesic (unless £ = 1). Using the 
scaled length gs in the denominator would entail a bias towards £ = 1. Each geodesic 
ys will almost surely not correspond to a streamline (unless both endpoints are on the 
streamline), which implies that there exists a t € (0, 1) such that the tangent vector 
Y(t) has a component in the direction of a non-principal eigenvector of the diffusion 
tensor. Since this component scales with e~', the connectivity would vanish for 
€ — 0 even if the geodesic remains (approximately) the same curve. Instead, the € 
control parameter merely serves to single out an optimal, metric-compatible geodesic 
through a suitable anisotropic scaling of the metric tensor. The connectivity criterion 
for the latter is based on data evidence, viz. average apparent diffusivity, which should 
not involve £. Connectivity C (y) is thus some average measure of diffusivity along 
Ys. Parameter values 

e* € argmax C (ys), (7) 

eels, 1] 


for some fixed 0 <6 < 1, at which locally > optimal connectivity is attained, are tract- 
specific sharpening parameters, affected in a still unknown manner, by underlying 
microstructure. The corresponding tracts y» are called optimal tracts. 

As illustrated in Fig. 2 for the Fiberfox-reconstructed Fibercup phantom with 
added Riccian noise [40] (cf. Fillard et al. for a detailed description [13] of the original 
Fibercup) only optimal tracts are retained. Since it is known that the inverse metric 
has certain shortcomings (e.g. see [24]), we also employed our automated procedure 
in combination with one of the other metric proposals in the literature, namely the 
adjugate metric [21, 22]. We chose this one, because it is derived from first principles 
of the underlying diffusion process and in addition it’s very easy to implement. In 
Fig. 2, the optimal tracts found for the inverse (bottom-left) and adjugate DTI metric 
(bottom-right) emerge for different optimal parameter values, but are, remarkably, 
virtually indistinguishable. As the adjugate DTI metric is essentially a local isotropic 
scaling of the inverse DTI metric, it is a noteworthy feature that we are able to obtain 
essentially the same tracts in both cases with a family of metrics characterised by a 
single, global parameter. After all, the governing geodesic equations for the inverse 
and adjugate are not the same. In practice, a broad range of parameter values (the 
high plateaux in the graphs) can be used to single out essentially the same tracts, 
making this procedure robust with respect to the precise choice of £. The experiment 
shows that, in all cases and by virtue of proper placement of endpoints, correct tracts 


5 We define argmax, f(x) = {x | f(y) < f(x) for all y in a neighbourhood of x}. 
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Fig. 2 (Top-left) Ground truth tracts (for various seed points); illustration adapted from Fillard 
et al. [13]. (top-right) Tracts obtained by using the unscaled inverse (red), respectively adjugate 
(blue) DTI tensor as the default metric for geodesic tractography. Both metrics produce shortcuts 
(false positives) instead of following the U-shape fibers, leaving false negatives. (middle-left) Per- 
tract connectivity for the -parameter unfolding of the inverse DTI metric, g= D™. (middle-right) 
Idem for the adjugate DTI metric, g =det D D™Y. Colors correspond to those used for the tracts in 
the bottom two figures. The red dots indicate the tract-specific parameter values £* € (0, 1] for the 
metrics used for each optimal tract. This optimal parameter is not affected by a suitable (global) 
normalization of g, meaning that the absolute value of the connectivity is inherently meaningless. It 
is, however, the relative value that determines the optimal parameter. Note the logarithmic scale for 
£. (bottom-left) Optimal results obtained by automatic optimization for inverse DTI metric ansatz. 
(bottom-right) Optimal results obtained by automatic optimization for adjugate DTI metric ansatz. 
The latter two results are virtually indistinguishable, but optimality is obtained at different control 
parameter settings. 
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Fig. 2 (continued) 


are found automatically, and, in particular, that shortcuts do not occur. To assess the 
robustness to placement of endpoints, in Fig. 3 the optimal tracts are computed for 
pairs of perturbed endpoints, demonstrated for the adjugate DTI metric. 

Low connectivity typically corresponds to shortcuts. Figure 4 shows optimal tracts 
for each pair of corresponding (ground truth) seed and target points (top), as well as 
connectivities for optimal tracts between a selection of seed point and all remaining 
target points (bottom). For each pair of possible seed/target points, the optimal tract 
was determined by Eq. (7). Figure 4 shows two representative cases: one in which 
the stipulated ground truth yields the highest connectivity and a second one in which 
it is among the top but is still dominated by another tract. 

Multiple local maxima of C(y,) generally exist, occurring at € € {e}, ..., eg} say, 
corresponding to k plausible tracts (recall Eq. (7) and footnote 5) that invariably 
include the stipulated ‘ground truth’, although the latter does not necessarily corre- 
spond to the global maximum, recall Fig. 4b. Figure 5 illustrates this case by another 
simulation (based on Fiberfox [40]), emblematic of an ambiguous configuration of 
kissing or crossing fiber tracts . Both interpretations are indeed confirmed, in the sense 
of corresponding to local optima of C(y,). Our current coarse-to-fine implementa- 
tion, however, cannot handle scale space bifurcations of geodesics automatically, 
and, due to our default initialization by the Euclidean geodesic, automatically zooms 
in on the kissing tract. The crossing tract requires a (simple) manual edit to overrule 
the default initialization and encourage convergence to another solution. This edit 
and its effect is shown in Fig. 5, and further illustrated in Fig. 6. Here the default ini- 
tialization by the Euclidean geodesic has been replaced by a quadratic Bezier curve 
forced to pass through a manually selected inclusive ROI called an AND gate (green 
circle), while connectivity maximization for this control point has been constrained 
to this neighbourhood. Note that, for the inverse as well as the adjugate diffusion 
tensor ansatz, Fig. 6, boundary extrema y, are found near ¢ = 1, illustrating the same 
inclination to produce shortcuts as y; for the unscaled metric. Closer to the optimal 
é* € argmaXsejs,1] C (Ve) we observe correct convergence, though. 
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Fig. 3 For each endpoint of the optimal green curve (cf. Fig. 2) the red points are generated 
from a multivariate normal distribution centered around the endpoints with covariance matrix X = 
diag(0.5, 0.5). The blue tracts are the optimal tracts for pairs of such perturbed endpoints by using 
the adjugate DTI metric ansatz. The green curve has a connectivity value of 2.749 and the mean 
connectivity of all perturbed tracts is 2.708 with a standard deviation of 0.037. Visually, the green 
curve is a good representative of this bundle of tracts as a whole and its connectivity is approximately 
one standard deviation away from the mean. 


In Fig. 7, the left Cortical Spinal Tract of the ISMRM 2015 Tractography challenge 
data [35] is used to illustrate the operational procedure on a 3D dataset. Quantitative 
measures used in the challenge are the overlap (OL), overreach (OR) and F; scores, 
see Côté et al. [9]. OL is defined as the number of voxels through which both the 
reconstruction and the ground truth pass, normalized by the number of ground truth 
voxels. OR is the number of voxels through which the reconstruction passes but the 
ground truth does not, normalized by the number of reconstructed voxels. These two 
quantities can be interpreted as the true positive rate and the false discovery rate. 
The F score is the harmonic mean of OL and 1 — OR. Our tractography results in 
OR = 0.324, OL = 0.375, leading to Fı = 0.427, ranking it in the top 40% of the 
submitted challenge results based solely on F; scores. It is particularly notewor- 
thy that this DTI-based procedure has a performance comparable to HARDI type 
methods, outperforming other DTI-based methods applied to the CST [36]. 
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Fig. 4 (Top) Seed and target point labeling: s; = start of tract, e; = end of tract; the attached number 
ie€{l1,..., 7}, refers to the tract label. (bottom) Box-and-whisker plot of connectivity in the adjugate 
DTI metric versus seed point. For the seed points sı and s2 the boxplot above it shows the optimal 
connectivity of that seed point to every other target point s; and e;. Endpoints that yield the highest 
connectivity are indicated on top on the boxplot. The red dots indicate the connectivity value for 
the stipulated ground truth, i.e. the tract running from s; to e;, for each tract label. 
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Fig. 5 Simulation with kissing/crossing fibers. Seed and target points are invariably taken to lie 
in upper and lower left corner. Note that there are two plausible solutions. The left two figures 
pertain to the (connectivity pruned) e-family induced by the inverse diffusion metric, the right two 
figures to that of the adjugate diffusion metric. In either case, our automatic procedure only finds 
one solution (blue curves). (top) The equally viable crossing scenario is due to the coarse-to-fine 
scheme with default initialization (red dotted Euclidean geodesic in the upper two figures). (bottom) 
By employing a restricted neighbourhood search region (AND gate, green circle) with suitable 
initialization (red dotted quadratic Bezier curve), the algorithm converges to the blue crossing 
geodesics in the bottom two figures, resolving the false negative result. 
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Fig. 6 Geodesics found by unfolding of the Riemannian metric family gẹ, Eq. (7) in combination 
with connectivity optimization, Eq. (6) for inverse (left) and adjugate (right) diffusion tensor ansatz 
g=g , the AND gate of Fig. 5 (green circle). Initialization is given by the quadratic Bezier curve 
in Fig. 5. Red-to-blue rainbow color coding shows the connectivity evolution as a function of €, 
with red indicating low connectivity tracts and blue indicating the (unique) connectivity optimized 
result for e* =argmax¢ers,1] C (Ve). 


4 Conclusion and Discussion 


We have proposed a modified Riemann-DTI geodesic tractography framework. The 
modification entails the embedding of a DTI induced metric into a family of metrics 
designed for the purpose of tractography, furnished with an operational scheme for 
tract-specific optimization of metric and geodesic(s) for any given pair of endpoints 
based on a nonlinear connectivity functional. 

In a simple experiment based on the Fibercup simulator we conclude that all local 
connectivity maxima correspond to plausible tracts that consistently follow high 
diffusivity pathways and avoid, as much as logically possible, inconsistent regions. 
For instance, despite a rather simplistic connectivity measure, all ground truth tracts 
are reconstructed provided seed and target points are (roughly) correctly placed, i.e. 
in such a way that a plausible fiber does indeed exist, recall Fig. 2. The robustness 
of our procedure with respect to the placement of endpoints is demonstrated in 
Fig. 3, where tracts reconstructed using slightly perturbed seed and target locations, 
remained in close proximity to the original one. 

It is essential, in this respect, to appreciate the ill-posed nature of tractography. 
Although the generation of diffusivity patterns from a given tract configuration is 
robust (albeit model-dependent), this process is not invertible. The equivalence class 
of tract configurations consistent with a given diffusivity pattern may contain many 
“‘metamers’, so to speak. The actual power of the proposed method is precisely its 
ability to generate multiple representatives from this equivalence class, so that no 
a priori bias is introduced. In clinical practice this could help an expert to reject 
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Fig. 7 Comparison of the Left Cortical Spinal Tract (CST) between the ground truth (red) and 
connectivity-optimal geodesics according to Eqs. (6) and (7) in which the base metric is the adjugate 
DTI-metric. The geodesics are color-coded according to their connectivity value, which increases 
from green to blue. The depicted views are labeled with S(uperior), A(nterior) and L(eft) to determine 
the plane and orientation of the cross-section. On visual inspection of the multiple views of the CST, 
the optimal geodesics lie close to the ground truth. Using more refined connectivity measures or a 
criterion to remove single tracts, the results can be improved. This can relieve us from those spurious 
tracts that are seemingly leaving the bundle and clearly have a lower connectivity than the rest of 
the tracts. A selection of 3000 ground truth tracts were extracted from the CST to determine seed 
and target points for the geodesics. 


Riemann-DTI Geodesic Tractography Revisited 239 


false positives (if not eliminated automatically), based on data-extrinsic anatomi- 
cal insight, in favour of selecting one of the remaining plausible candidate tracts, 
i.e. without being confronted with false negatives. Ideally such candidates present 
themselves, viz. as one of the local optima actually found by our automatic scheme. 

False negatives may nevertheless occur if local maxima are missed due to inade- 
quate numerical optimization. We have indicated how this problem may be mitigated 
through a simple manual edit by an expert. The delineation of an AND gate through 
which one believes tentative tracts of interest should pass, or the sketch of a coarse 
initialization curve between the endpoints, overruling the default straight line seg- 
ment, both make it more likely to drive numerical optimization to the correct local 
maximum. 

To prevent scaling of the diffusion tensors in a non-sensical manner, their eigen- 
values are only affected whenever an FA threshold was exceeded in order to ensure 
a well-defined main eigendirection of the diffusion tensor. This treshold was based 
on experiments with the Fiberfox-reconstructed Fibercup phantom, but needs to be 
adjusted for tractography on real data. This ad hoc approach should be considered 
a first stepping stone towards a more refined technique combining the scaling of 
eigenvalues with the anisotropy of the tensor and the noise level of the data. Highly 
anisotropic tensors have a relatively large first eigenvalue and may tolerate a more 
severe scaling, since they provide more evidence for an articulated orientation. Since 
many low FA regions contain complex fiber structures, not scaling the DTI tensors 
there may influence the shape of the geodesics. In these areas we do not expect to 
get accurate results, since DTI is not able to resolve complex fiber structures ade- 
quately. However, scaling of the tensors enhances any noise present in the data and 
may increase the number of false positives, especially when the signal-to-noise-ratio 
is very low. 

We stress that ‘most likely’ fiber candidates are obtained for any pair of endpoints 
in terms of local optimality of connectivity. Recalling Fig. 4, note that even though 
tract s2-€2 has a high connectivity value, maximal connectivity is attained by the 
tract So-ss. This effect can be ascribed to the construction of the family of metrics 
ge as well as the conservative nature of the connectivity measure in Eq. (6). At the 
intersection of two bundles the diffusion tensors are (nearly) isotropic, leading to 
similar diffusivities in all directions. This isotropy (FA < 0.15) prevents anisotropic 
rescaling of the metric, as explained. Moreover, as the connectivity measure only 
accounts for average diffusion along a tract and does not penalize curvature, a bend 
in the tract does not decrease its connectivity, as the latter is based on diffusivity, not 
geometry. These effects combined result in a connectivity-optimal tract which differs 
from the ground truth one. This needs to be remedied by using more sophisticated 
connectivity measures, which will be application dependent and may require prior 
knowledge about the bundles of interest. Our generic connectivity measure has been 
motivated mainly by our desire to preserve true positives, not as a criterion to remove 
false positives, allowing a modular approach. Additional criteria for pruning false 
positives connecting a fiducial seed point to a volumetric target region (possibly the 
whole brain) will be studied in future work, including appropriate extensions of the 
proposed coarse-to-fine tractography algorithm. 
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In the ISMRM Tractography challenge reconstructed tracts are interpreted as 
centerlines of a fiber with a thickness/resolution of 1 mm isotropic voxel. The OL, 
OR and F scores were calculated based on such 1 mm voxels [36]. The trac- 
tography, however, was performed on the diffusion weighted images consisting of 
2 mm isotropic voxels. This way we can only resolve detail up to the grid scale of 
2 mm and quantification measures should take this resolution difference into account 
when using the affine transformation to convert between the two different spaces. 
By downscaling this minimal resolution of 1 mm to 2 mm we increase the F score 
to 0.526, ranking in the top 20% (as the only DTI based algorithm). 

In this experiment details could be resolved up to grid scale, but if one would 
consider the effects of noise (e.g. in the acquisition process), this may lead to a 
different effective minimal resolution, determined by grid scale and noise level. The 
influence of different kinds of noise (e.g. acquisition noise, seeding errors) needs to 
be addressed in future studies in which the control-triad framework is combined with 
uncertainty quantification. 

These experiments confirm our main conjecture that the potential of DTI has not 
yet been fully exploited in tractography. 
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1 Introduction 


1.1 Background 


Magnetic Resonance Imaging (MRI) allows us to probe structural anisotropy of 
tissue in vivo by studying the magnetic properties and translational motion of for 
instance hydrogen protons. In the human body, hydrogen is naturally abundant in 
various compounds, with the highest MR signal amplitude detected from hydrogen 
in water molecules. Hydrogen protons possess spin angular momentum, which is an 
intrinsic quantum property that allows the occurrence of magnetic interactions and 
resonance. When placed in a magnetic field, there is a slight preference for spins 
to be aligned with the field, resulting in a net magnetisation M aligned with the 
main magnetic field Bo [1]. Upon the application of a radiofrequency field at the 
Larmor frequency B,, M can be tipped out of alignment with By; most commonly 
into the perpendicular plane. The measured signal is the result of the ensemble of 
spins precessing coherently in the plane, and signal loss (decay) occurs when such 
coherence reduces. 

This signal decay, which results from a progressive loss of coherence of pre- 
cessional phase, i.e. dephasing, is also called spin-spin relaxation. The spin-spin 
relaxation rate is usually denoted by R? = 1/72, where T is the time taken for the 
magnetization to decay to 1 /e of its initial value [2, 3]. In addition to the irreversible 
spin-spin relaxation, dephasing can be caused by local variations in the magnetic 
field, a reversible process if the spins are static. Such local variations in the magnetic 
field arise from a difference in interaction of different substances with the magnetic 
field (susceptibility effects). If the spins experience Brownian molecular motion, they 
will experience various magnetic field strengths, and their dephasing due to local field 
changes will be effectively irreversible. Hence, the measured apparent T> of signal 
decay will no longer be purely induced by dephasing due to spin-spin interactions, 
but will depend on the amplitude and spatial characteristics of the local field vari- 
ations, and the mean displacement of molecules per unit of time due to incoherent 
motion. 

In diffusion-weighted MRI, additional (and typically much stronger) magnetic 
field variations are induced intentionally by applying magnetic field gradients which 
cause the strength of the main magnetic field Bo to vary linearly in space [4—6]. As 
such, the Brownian molecular motion can be encoded in the signal in a controlled 
way. 

The interplay of susceptibility and diffusion effects leads to the anisotropy of tissue 
being reflected in different MRI contrasts and hence the combination of contrasts can 
give a more complete picture of tissue microstructure. In the following paragraphs 
we will describe anisotropy in the brain and these processes in more detail. 


Structural anisotropy of human brain tissue. The dominant tissue exhibiting 
anisotropy in the brain is the white matter (WM). It is predominantly composed 
of the long extensions of neuronal cells—the axons, which are are grouped into 
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fibre bundles and inter-connect different areas of the brain. The main function of 
axons is to transmit electric impulses between and within brain areas. Axons can be 
insulated by a myelin sheath which is formed of lipid chains and allows a faster sig- 
nal transmission. The size, density, length of the axons and their myelination levels 
vary with age and may alter with pathology. Therefore, investigation of the white 
matter microstructure is of high importance in understanding the functionality of the 
healthy brain, but also in studying the mechanisms of normal/abnormal development 
and pathology. 


Diffusion effects. Measurements of Brownian molecular motion in biological tissue 
can reflect not only the temperature and the viscosity of the medium it is occurring 
in, but most importantly will be sensitive to the underlying geometry and anisotropy 
of the tissue. For instance, in the brain white matter water molecules can propagate 
much easier along fibre bundles than perpendicular to them because of obstacles 
such as the cell membrane and myelin [7]. This property can be used to estimate the 
main orientation of fibre bundles and their virtual reconstruction by means of fibre 
tractography [8, 9]. Additionally, if water molecules are trapped inside the axon 
and cannot penetrate the boundaries (restricted diffusion), the mean displacement 
perpendicular to the axon will be similar to its diameter at long diffusion times. The 
diffusion of water molecules which reside outside axons is commonly thought of as 
not being fully restricted but hindered. An example of the differences between the 
movement of water molecules residing inside and outside of axons is visualised in 
Fig. la. 

In MRI, Brownian motion of water molecules is most commonly encoded using a 
pair of pulsed magnetic field gradients, a dephasing and a rephasing gradient [6] (Fig. 
2). If spins are stationary, these gradients would have no additional effect on the signal 
decay. However if spins change their positions during or between the application of 
the gradients, the rephasing will be incomplete which will result in signal loss. The 
signal loss due to diffusion can be enhanced by increasing the magnitude of the 
gradients, the time during which the gradients are on, and/or the time between the 
gradients. The strength of the diffusion weighting is described by the b-value; a 
parameter which combines the information on the diffusion gradient strength and 
timings. 


Magnetic susceptibility effects. Any material placed inside a strong magnetic field 
interacts with it—it can become magnetised itself. The proportionality constant link- 
ing magnetic field strength and the magnetisation induced inside the material is called 
magnetic susceptibility. At the boundaries between materials with different magnetic 
properties the magnetic flux density is spatially inhomogeneous and its distribution 
will depend on the boundary orientation to the magnetic field and the difference 
in susceptibility between the materials. Additionally, the magnetisation induced in 
some materials may also depend on the orientation of the sample to the magnetic 
field, i.e. the magnetic susceptibility of those materials is anisotropic. 

As mentioned above, the nerve fibres in human brain are insulated by myelin 
sheath. Myelin is more diamagnetic than water, i.e., it is repelled more strongly by 
a magnetic field. Additionally, several studies suggested that the magnetic suscepti- 
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Fig. 1 A white matter nerve fibre can be modelled as a hollow cylinder composed of a myelin 
sheath. a Brownian motion prependicular to the nerve fibre is restricted inside the cylinder (the 
molecules are “trapped” inside the cylinder) and hindered outside of it (the mean displacement of 
the molecules per time-point is larger, but nevertheless lower than ‘free’ water). b The myelin sheath 
is more diamagnetic than the surrounding tissue and perturbs the local magnetic field. The field 
perturbation is inhomogeneous and depends on the nerve fibre orientation to the magnetic field Bo 
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Fig. 2 Pulsed-gradient spin-echo sequence: the echo time TE is defined as the time between the 
radio-frequency pulse and the centre of the spin-echo. The diffusion-weighting strength, the b-value, 
can be calculated as b = y7g767(A — 6/3) 


bility of the myelin sheath has an anisotropic component [10-12]. It has been shown 
that the signal decay from white matter varies as a function of orientation to Bo 
[13-20] and can be well explained using a hollow cylinder fibre model (Fig. 1b) [17] 
and is often represented by a 3-pool model [18, 21, 22]. 

The strongest contribution to this signal anisotropy arises from the water trapped 
between the layers of myelin sheath, called the myelin water. However, the myelin 
water signal decays very quickly, on account of its short T} (~10-30 ms), and is 
usually negligible at the echo times used in a typical diffusion-weighted MRI exper- 
iment. Nevertheless, it has also been observed that signal decay rates may still be 
orientation-dependent even without this myelin-water component [13, 20, 23-25]. 


Magnetic Resonance Imaging of T2- and Diffusion Anisotropy ... 251 


1.2 Scope of This Work 


In myelinated white matter it has been reported that T¥ (where 1/ T = 1/T2 + 1/T; 
and T; can be understood as capturing the reversible effects of field inhomogeneities) 
depends on the orientation of the fibre with respect to Bp due to microscopic sus- 
ceptibility effects [13, 16, 18, 19, 26]. Orientation-dependence of T) was also 
reported recently [20, 23, 24, 27] and could potentially characterise microscopic- 
susceptibility more reliably and reflect effects related to axon diameter [24, 28]. 
Experiments designed to probe relaxation-anisotropy commonly involve reorienting 
the head inside the scanner, and are thus challenged by unintended signal-to-noise 
ratio (SNR) variations across orientations caused by differences in proximities to the 
receiver coil, and by increased susceptibility to motion and artefacts due to patient- 
discomfort. In this work, we re-purpose a tiltable RF coil (originally designed for 
patient comfort) to investigate 7>-orientational dependence within the context of a 
diffusion-7> correlation experiment. The coil can be tilted around the left-right axis 
by 0°, 9° and 18° to Bo, which: (1) minimises patient-discomfort and thus improves 
reliability; (2) offers a new degree of freedom as tilting around the left-right axis 
is otherwise difficult to achieve; (3) fixes the coil-to-brain distance across orienta- 
tions and thus reduces SNR variations; and (4) increases the reproducibility of the 
experiment. 

Finally, instead of studying global variations across the whole brain volume, we 
adopt an along-tract profiling tractometry approach (i.e., which is the mapping of 
measures along pathways reconstructed with tractography [29, 30]) to assess spatial 
variations in more detail. 


2 Methods 


2.1 Data Acquisition 


The study was approved by the Cardiff University School of Psychology Ethics Com- 
mittee and written informed consent was obtained. Two healthy volunteers (female, 
30y.) were scanned on a 3 T 300mT/m Connectom scanner equipped with a mod- 
ified 20-channel head/neck tiltable coil (Siemens Healthcare, Erlangen, Germany). 
Each subject was in supine, head first position and the direction of the magnetic 
field Bọ was along the superior-inferior radiological axis. MRI data were acquired 
in the default (0°) and tilted (18°) orientations of the tiltable coil (Fig. 3a, b). One 
of the subjects underwent a second scan in the default head orientation to examine 
test-retest variability. 

Diffusion-7> correlation data were acquired using a pulsed-gradient spin-echo 
echo-planar-imaging (PGSE-EPI) sequence [6] (Fig. 2), with different echo-times 
TE to probe T, and diffusion-weighting strengths or b-values to probe diffusion, 
(Fig. 3c). The timings of the diffusion encoding gradients were fixed for all echo 
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Fig. 3 a The coil in default (0°) and tilted (18°) position. b b = 0s /mm? images at different TE. 
c Acquisition parameters for the diffusion-correlation experiment 


times. The signal in voxel x can then be denoted as S,(b, TE). Additional bọ-images 
were acquired in the halfway-tilted (9°) position (not shown). Remaining parameters 
were repetition time TR 3.5 and voxel size 3 x 3 x 3 mm°. 


2.2 MRI Signal Processing 


The diffusion-7> data were preprocessed to correct for subject motion, eddy current 
effects, Gibbs ringing and gradient non-linearities [31-33] for each subject and each 
head orientation. Spatial correspondence between the tilted and default head orienta- 
tions was obtained in two ways: 1) by nonlinear registration [34] to the halfway-tilted 
(9°) space, and 2) by a tractometry approach in native space. 

The tractometry [29] approach in native space of each head orientation relied 
on the quantitative mapping of measures along reconstructed brain pathways. First, 
fibre orientation distribution functions (f(ODFs) [35, 36] were estimated at each voxel 
using multi-tissue multi-shell constrained spherical deconvolution (MSMT-CSD) 
[37]. For each coil position, peaks were extracted from the resulting fODFs and used 
as input to perform streamline tractography on the TE = 54 ms data. Bundles were 
automatically segmented [38], a representative core-streamline was computed [39], 
and the bundles were subsequently subdivided [40, 41] into n = 20 segments (s) 
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Fig.4 Extracted brain pathways of interest from subject 2. Bundles were segmented into 20 sections 
(0: ligth-blue, 20: dark-brown). CC: corpus callosum, UF: uncinate fasciculus, CST: cortiospinal- 
tract, IFOF: inferior fronto-occipital fasciculus, ILF: inferior longitudinal fasciculus. Visualisation 
was performed using FiberNavigator [43] 


(Fig. 4). Voxels with a single-fibre-population were identified [42] and assigned to 
si if their location was inside the segment and their orientation was within 30° of the 
orientation of the core-streamline in that segment. Note that only tract-segments with 
minimal fanning (assessed visually) were considered. T) values were then profiled 
for each bundle independently by taking the mean and standard error of the mean 
within each tract segment. 


2.3 Estimation 


SNR estimates were obtained from the background of the images acquired at TE = 
54ms and b = 0s/ mm? for both the (0°) and (18°) coil-orientations [44]. The voxel- 
wise T) was estimated from the bo-signals as S(0, TE) = S(O, Oe TER, using a 
nonlinear least-squares trust-region-reflective algorithm in Matlab. 

Fibre orientation 0 to the main magnetic field Bọ was estimated for the vox- 
els with single fibre population. With prior knowledge of 0 orientation-dependent 
(anisotropic) and -independent (isotropic) components of Rz can be estimated as 
follows [20, 45]: R2 (0) = R2 isotropic F R2 anisotropic : sinf 0. 


3 Results 


Signal-to-noise ratio. The estimated noise standard deviation was similar for the two 
coil-orientations. Figure 5a compares the signals in the default and the tilted position 
after registration to the common space; the signal (and thus SNR) between coil- 
orientations varied within a similar range as the range seen between test-retest scans in 
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the same orientation. Globally, the images are aligned but some local misalignments 
can still be observed. 


Voxel-wise 7>-estimates. Figure 5b highlights differences in estimated T; for differ- 
ent coil-orientations. Globally, the difference in 7>-values estimated from the data 
in default and tilted head orientations is larger than or equal to T2-values estimated 
from test-retest scans in default head orientation for subject 1 (Fig. 5b, left half). 
Local differences in T) can be observed between the default and tilted position, as 
indicated by red arrows in the 7>-maps in Fig. 5b, for example. The inverse 7-values 
in white matter mostly range between 9s~! and 21s~! for both subjects and head 
orientation. 
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Default Tilted a Difference 5 Default Tilted Difference 
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Fig. 5 a b= 0s/mm? image for the default (0°) and tilted (18°) orientation registered to the 
halfway-tilted (9°) space, and their difference (tilted—default). ô is the estimated standard deviation 
in the background of the image. b Estimated T> for the default (0°) and tilted (18°) orientation 
registered to the halfway-tilted (9°) space, and their difference. Red arrows indicate regions of 
visible difference 
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Fig. 6 Relaxation rate 1/7 as a function of the fibre-orientation 0 with the main magnetic field, 
in the default (left) and tilted (right) position. Each point represents a single-fibre-population voxel 
and is colour-coded according to its orientation (red, blue, and green correspond to the left-right, 
superior-inferior and anterior-posterior axis, respectively). The black line represents a model-fit of 
1/T as a function of @ as described in previous literature and references therein [13, 19, 20] 


Global WM isotropic and anisotropic 7>- stimates. The total isotropic (orientation- 
independent) component of relaxation rate of Ro = 1/7) in WM was estimated at 
13.7 +0.2s7! and 13.6 + 0.2s7!, in default and tilted head orientations, respec- 
tively, for one of the subjects (Fig. 6). For the same subject the total WM anisotropic 
components of the inverse T were 1.6 + 0.25 s7! and 1.7 + 0.25 s7}, for default and 
tilted head positions, respectively. The range of 7-values in white matter and their 
total isotropic and anisotropic components are consistent with previously reported 
values in [20, 45]. 


Tractometry analysis. Figures7, 8 and 9 show the along-tract profiles of the esti- 
mated T, (top plot) and angle w.r.t. Bọ (bottom plot) for different tracts. Globally, 
the angular profiles show comparable characteristics between subjects in the default 
and tilted position. For Subject 1, the angular profiles remain similar in the default 
and default-retest acquisition, except for the inferior parts of the corticospinal tract 
(CST). 

Profiling of T in the CST, which runs along the z-axis (i.e., inferosuperior), 
reveals a significant increase in 7, for example in segment 16 which experiences 
a change in orientation w.r.t. Bọ from ~35° in the default position to ~55° in the 
tilted position (Fig. 7). This is in the regime where the derivative of the relaxation 
rate 1/ T as a function of angle is the largest (Fig. 6). The uncinate fasciculus (UF) 
shows a noisier pattern, likely because the number of single-fibre voxels per segment 
is generally lower (~5—10 in the UF compared to ~15—20 in the CST). 
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Fig.7 Tractometry results (mean and standard error) for the corticospinal tract (CST, top plots) and 
the uncinate fasciculus (UF, bottom plots). 7>-values and fibre orientation to Bo are plotted against 
segment numbers for each fibre tract and subject. Colour bar indicates tract-segment location, 
corresponding to the colour-encoding on the visualised tracts. Note that only tract-segments with 
minimal fanning (assessed visually) were considered 


In the inferior longitudinal fasciculus (ILF) (Fig. 8) a global decrease in angle 
w.r.t. By from default to tilted position leads to an overall, yet subtle, increase in 7. 
In the inferior fronto-occipital fasciculus (IFOF) the pattern is less clear. 


In the callosal midbody the angle w.r.t. Bo remains relatively unchanged and so 
does T>. 


4 Discussion 


In this work we have incorporated a tiltable coil in T>-diffusion-correlation experi- 


ment to modulate the white matter fibre orientation with respect to the main magnetic 
field Bo. 
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Fig. 8 Tractometry results (mean and standard error) for the inferior fronto-occipital fasciculus 
(IFOF, top plots) and the inferior longitudinal fasciculus (ILF, bottom plots). T>-values and fibre 
orientation to Bo are plotted against segment numbers for each fibre tract and subject. Colour bar 
indicates tract-segment location, corresponding to the colour-encoding on the visualised tracts. Note 
that only tract-segments with minimal fanning (assessed visually) were considered 


We observed changes in the 7>-tract-profile after the participants’ heads were 
re-oriented in the scanner, with up to ~10ms difference in 7>-values between the 
default and the tilted head orientation. The test-retest 7-tract profiles in the default 
head position are more similar to each-other than to the tract profiles in the tilted 
head position, as for example evident from the results for the CST and IFOF. These 
initial results suggest variation of T as a function of fibre orientation to Bo and that 
tilting the participant’s head by 18° can reveal those variations. 
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Fig. 9 Tractometry results for the genu of the corpus callosum (CC2). 7>-values and fibre ori- 
entation to Bo are plotted against segment numbers for each subject. Colour bar corresponds to 
tract-segment location, and this is colour-encoded on the tracts themselves. Note that only tract- 
segments with minimal fanning (assessed visually) were considered 


4.1 Incorporating Tiltable Coil in Ty-diffusion correlation 
experiments 


Robustness of experimental setup when using a tiltable coil. The tiltable coil 
allows us to control the pitch orientation of the participant’s head. We have demon- 
strated an overall test-retest similarity of estimated fibre-tract orientations in the 
default position, whereas there were clear differences with the tilted head position. 
However, some differences in fibre orientation of the CST between the test and retest 
could be observed, particularly in the inferior segments, while differences in fibre 
orientation in the IFOF tract were less obvious. Given the apparent stability of the 
IFOF tract profiling and insensitivity of fibre orientation to the head rotation around 
Bo in the default head position (yaw), the test-retest differences in the CST could 
be a result of an additional roll head orientation with respect to the coil between the 
two scans. Such additional differences in orientation could be mitigated by further 
restricting head position within the coil, e.g. additional padding or performing the 
experiments with differences in tilt immediately after each other without a break. 
Whereas this introduces a confound in assessing test-retest variability, the estima- 
tion of fibre-orientations is done in each coil orientation independently and as such 
we do not expect this to be detrimental in the overall assessment of T>-anisotropy. 


Range of orientations. By re-purposing a coil that was designed to maximise patient 
comfort in clinical situations means that the range of coil orientations was limited. A 
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larger range of orientations would allow the 0 versus 1 / T relationship to be elucidate 
more robustly. Nevertheless, these initial experiments demonstrate the utility of this 
hardware design for uncovering orientational-anisotropy effects in vivo. 


4.2 Origin of T,-Contrast and -Anisotropy in WM 


The water signal from each WM voxel is a superposition of the signal from different 
water pools (e.g. intra- and extra-axonal, and at short TE also myelin water), therefore 
the macroscopic 7-values measured in this study could be approximated as the 
weighted average of the 7>-values from each of the two compartments. This means 
that the macroscopic T) will depend on individual T>-values of the compartments, 
but also on the relative contribution to the signal. As such, tracts with the same 
apparent T, but differences in signal fractions and 7>-values between the intra- and 
the extra-axonal compartments could exhibit very different apparent orientational 
anisotropy. 

A clear separation of macroscopic 7-values in some tracts and segments between 
the default and the tilted head orientations suggests that macroscopic T> is also a 
function of orientation to Bo. Fibre tract re-orientation in the magnetic field will cause 
local changes in the local magnetic field due to differences between the magnetic 
susceptibility of the myelin sheath and the surrounding tissue. Following the hollow- 
cylinder model [26], the magnitude of these microscopic Bo-field perturbations is 
expected to be larger in the extra-axonal compartment and, in combination with 
molecular Brownian motion, will cause an additional faster relaxation in the extra- 
relative to the intra-axonal compartment [46]. Future work will explore the effects 
of fibre orientation on individual compartments [47, 48]. 


4.3 Considerations in Data Processing 


Tractometry versus image registration In this work we adopted tract profiling 
for comparing the 7>-diffusion-correlation data in default and tilted head positions, 
instead of a voxel-based analysis; image registration was used for visualisation pur- 
poses only. Voxel-based analysis is known to suffer from confounds related to mis- 
registration and data interpolation, an effect we visually observed in our study likely 
amplified by the large imaging voxels (3 mm isotropic) and imperfect correction for 
geometric distortions due to gradient nonlinearities. With the tractometry approach, 
spatial correspondence between coil orientations was established while keeping the 
data in native space, and each segment consitutes information from multiple voxels 
improving robustness to noise. We were able to reproduce 7>-values from test-retest 
along the tracts. 
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Correction for subject motion The individual images have been corrected for 
subject motion, which could also involve head re-orientation with respect to the 
magnetic field. Subjects who participated in this study were experienced MR scanner 
participants, therefore the maximum rotation around any axis rarely exceeded 1.5°. 
However, when considering the application of this method on less compliant subjects, 
subject motion could become a confounding factor. 


5 Conclusion 


Including a tiltable coil into the experimental set-up for diffusion-7> correlation mea- 
surements paves the way for a more reliable assessment of orientational T, depen- 
dence. Microstructural origins of the differences in T) could include differences in 
pathway properties (e.g. axon diameter) or susceptibility effects. T) orientational- 
dependence would furthermore impact analyses frameworks that assume constant Tz 
along pathways [49]. Voxel-wise comparison of T) from different head-orientations 
remains challenging due to complications in experiment setup, imperfect correc- 
tion for geometric distortions, and intrinsic scan-variability. Using a tractometry 
framework, we found indications of regional changes in T) upon tilting of the head. 
Studying this effect in a larger population is necessary to increase statistical power. 
In future work, the diffusion-7> correlation experiments can be used to study com- 
partmental T> orientation-dependence. 
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Abstract The placenta has a unique structure, which enables the transfer of oxygen 
and nutrients from the mother to the developing fetus. Abnormalities in placental 
structure are associated with major complications of pregnancy; for instance, changes 
in the complex branching structures of fetal villous trees are associated with fetal 
growth restriction. Diffusion MRI has the potential to measure such fine placental 
microstructural details. Here, we present in-vivo placental diffusion MRI scans from 
controls and pregnancies complicated by fetal growth restriction. We find that after 30 
weeks’ gestation fractional anisotropy is significantly higher in placentas associated 
with growth restricted pregnancies. This shows the potential of diffusion MRI derived 
measures of anisotropy for assessing placental function during pregnancy. 


1 Introduction 


Fetal growth restriction (FGR) is a condition where the developing fetus does not 
reach its full growth potential in-utero [13]. It constitutes a major pregnancy compli- 
cation and is associated with a high degree of fetal mortality, morbidity and life-long 
complications [19]. Early onset FGR (defined as that diagnosed before 32 weeks’ 
gestation) affects 0.5-1% of pregnancies and late onset FGR (diagnosed after 32 
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weeks’) affects 5-10% of pregnancies [6, 9]. Early diagnosis and close monitoring 
are essential to optimize the outcome for pregnancies affected by FGR. Recently, 
pharmacological treatments have shown promise for severe early onset FGR cases 
[31]. However, while routine antenatal monitoring utilising symphysis-fundal height 
and ultrasound measurements can identify a significant proportion of FGR cases 
[10], detection remains a major problem—only 31% of cases were diagnosed dur- 
ing a previous study [11]. Detection is crucial as FGR is a leading cause of late 
unexpected stillbirth [11]. 

Post-delivery histopathological analysis shows a significant degree of character- 
istic pathologies in FGR placentas [15, 33], emphasising its importance and involve- 
ment in the cascade of events leading ultimately to sub-optimal growth. However, 
while post-delivery detection or confirmation plays an important role in increasing 
knowledge and possible causes of FGR, it comes too late to influence clinical care 
for these pregnancies. Therefore, recent novel developments, mainly using Mag- 
netic Resonance Imaging (MRI) focus on studying the placenta during pregnancy to 
complement available antenatal screening. 


1.1 Placental Microstructure 


The placenta constitutes the key connection between mother and baby in utero and 
acts as the life support system for the growing fetus. Among its many functions 
are the exchange of oxygen and nutrients from the maternal blood circulation to 
the fetal blood, and the removal of waste products. The placenta comprises 10-40 
individual lobules each constituting one key exchange unit. The transfer relies on a 
delicate and dynamically evolving microstructure within these units, focused around 
the fetal villi and depicted in Fig. 1A. These tree-like structures originate from the 
umbilical cord and contain fetal arteries and veins. They are bathed in maternal 
blood that enters the intervillous spaces from the spiral arteries at the level of the 
basal plate. A thin membrane called the syncytiotrophoblast separates maternal and 
fetal circulation, and allows the transport of oxygen and nutrients through it. Sev- 
eral histopathological features are associated with FGR, including elongated villous 
trees without the appropriate branching patterns [21] as illustrated schematically in 
Fig. 1B. 


1.2 Placental MRI 


Specific challenges of in-utero placental MRI include motion, such as maternal 
breathing and fetal movements, various air-tissue interfaces such as amniotic fluid, 
abdominal fat and bowel gas, and the suboptimal position of the imaging coil with 
respect to the organ of interest, especially for placentas located on the posterior wall 
of the uterus [5]. Another important limitation specifically for diffusion MRI arises 
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Healthy villous tree FGR villous tree 
(schematic) (schematic) 


Fig. 1 Schematic representation of single fetal villous trees in the placenta. Fetal blood flows 
through the convoluted branching structures, allowing nutrient exchange with the surrounding 
maternal blood. Panel A: complex branching structure of a healthy placenta. Panel B: pathological 
branching structure, associated with prengancy complications such as fetal growth restriction 


from the safety requirement to reduce the acoustic sound level of the sequence to 
protect the fetal hearing. Therefore, typically the gradient slew rate is reduced, in 
consequence increasing the read-out length and thus the echo time resulting in lower 
Signal-to-Noise Ratios (SNR) [5, 16]. 

Diffusion MRI can reveal fine tissue microstructure details through sensitivity to 
the diffusion of water. By varying the strength and direction of diffusion gradients, 
the MRI sequence can be specifically tailored to microstructural features of interest. 
For example, sequences with a high number of distinct gradient directions can inform 
on tissue anisotropy and directionality. Such diffusion MRI sequences are commonly 
used to image white matter fibre tracts in the brain, however applications elsewhere 
in body exist, including in the placenta [3, 20, 27]. 


2 Methods 


Pregnant women, recruited as part of a larger cohort for the Placental Imaging Project 
(PIP), underwent an MRI scan between 20 and 40 weeks of gestation. Informed 
consent was obtained (REC 14/LO/1169) and the scan was performed on a clinical 
3T Philips Achieva (Best, The Netherlands) scanner using the 32-channel cardiac 
coil. All women were scanned in supine position under frequent monitoring of heart 
rate, saturation and blood pressure throughout the scan. Dedicated padding was 
provided to increase maternal comfort and verbal interaction was maintained. 

After initial structural T2-weighted sequences of the entire uterus and the fetal 
brain in multiple orientations, a BO map was acquired. An in-house developed tool 
for image based shimming was employed [12] to focus the shim on the placental 
parenchyma, avoiding air-tissue interfaces with bowel gas as much as possible. We 
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next acquired a diffusion-prepared spin echo sequence. For some participants we 
used a modified sequence which acquires additional gradient-echos after the initial 
Spin Echo (e.g. [28]), although in this study we only utilised the first spin echo mea- 
surement. Conventional Stejskal-Tanner diffusion preparation was performed, with 
a total of 65 b-value/b-vector combinations, optimized specifically for the placenta 
as previously described [17, 26], with three diffusion gradient directions at b = [5, 
10, 25, 50, 100, 200, 400, 600, 1200, 1600] s/mm?, eight directions at b = 18 s/mm?, 
seven at b = 36 s/mm”, fifteen at b = 800 s/mm?, and six b = 0 volumes. Further 
parameters include FOV = [300-340] x 320 x 84mm, TR = 7 s, SENSE = 2.5, 
halfscan = 0.6, resolution = 3mm? for scans with additional gradient-echos and 
2mm? otherwise. The total acquisition time was 8 min 30 s. Fat was suppressed with 
SPIR saturation pulses. The acquisition plane was coronal to the mother, chosen to 
assure that the in-plane direction coincides with the longest placental dimensions in 
mostly anterior and posterior placentas. 

To correct for motion, the diffusion weighted volumes were registered non-rigidly 
to a common template using the ANTs multivariate template construction tool with 
the cross-correlation similarity metric [1]. Subsequently, a region of interest (ROD 
comprising the whole placenta and adjacent basal placenta was manually segmented 
in all slices of the first b = 0 volume. We estimated diffusion tensor, mean diffusivity 
(MD), and fractional anisotropy (FA) maps—using all diffusion weightings—for the 
motion corrected scans using MRTrix [32]. 


2.1 Recruitment 


We include a total of twenty-nine participants in this study, who were categorised as 
follows. Sixteen women were normal uncomplicated control pregnancies; their out- 
comes were obtained and checked to ensure that no new diagnosis of pre-eclampsia, 
gestational hypertension, fetal growth restriction or gestational diabetes had occurred, 
and their birthweight was greater than the 5th centile (by INTERGROWTH-2 Ist). 
Seven women were diagnosed with fetal growth restriction, detected from antena- 
tal ultrasound assessments. We also include six women recruited as uncomplicated 
control pregnancies who gave birth to a baby under the 5th centile, but did not have 
a formal antenatal FGR diagnosis. Although these could simply be constitutionally 
small babies, they could also be undiagnosed FGR cases, so we hence analysed them 
as a separate cohort. Five of the six below the 5th centile had co-morbidities, such 
as chronic hypertension in pregnancy (CHTN), or pre-eclampsia (PE). The patient 
population characteristics are given in Table 1. 
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Table 1 Scan details for all participants. FGR denotes participants diagnosed with fetal growth 
restriction according to guidelines [13]. Under 5% denotes participants with no FGR diagnosis who 
delivered a baby weighting under the 5th percentile 


Participant ID Cohort Gestational age at scan 
1 Control 23.86 
2 Control 26.14 
3 Control 26.72 
4 Control 26.72 
5 Control 27.14 
6 Control 27.14 
7 Control 28.29 
8 Control 28.86 
9 Control 28.86 
10 Control 29.67 
11 Control 29.86 
12 Control 31.29 
13 Control 33.43 
14 Control 35.57 
15 Control 36.29 
16 Control 36.43 
21 FGR 22.0 
22 FGR 23.42 
23 FGR 28.57 
24 FGR 29.57 
25 FGR 30.85 
26 FGR 32.85 
28 (Scan 1) FGR + CHTN 30.71 
28 (Scan 2) FGR + PE 34.14 
29 Under 5% 21.29 
30 Under 5% 25.72 
31 Under 5% + CHTN 38.0 
32 Under 5% + CHTN 19.86 
33 Under 5% + PE 28.71 
34 (Scan 1) Under 5% + PE 31.42 
34 (Scan 2) Under 5% + PE 33.42 
3 Results 


Figures 2 and 3 display FA maps, ordered by gestational age (GA) at scanning 
time, for control and growth restricted participants respectively. We next examine 
the evolution of FA values over gestation (Fig. 4). Finally, we compare FA values 


268 P. J. Slator et al. 


Control, GA = 23.86 Control, GA = 26.14 Control, GA = 26.72 
FA 
1.0 
0.8 
0.6 
0.4 
Control, GA = 27.14 Control, GA = 27.14 
0.2 
l ee 
Control, GA = 28.29 Control, GA = 28.86 Control, GA = 28.86 
[] 
Control, GA = 29.67 Control, GA = 29.86 Control, GA = 31.29 
Tg. 


a 
Control, GA = 33.43 Control, GA = 35.57 Control, GA = 36.29 


ff 


Control, GA = 36.43 


Q 


Fig. 2 Fractional anisotropy maps for all control participants. The boundary between the uterine 
wall (higher FA) and placenta (lower FA) is clear in most placentas with gestational age less than 
30 weeks 


Anisotropy in the Human Placenta in Pregnancies Complicated ... 269 


FA FGR, GA = 22.0 


oe 


FGR, GA = 23.42 FGR, GA = 28.57 


08 


06 


04 


FGR + CH, GA = 30.71 
A FGR, GA = 30.85 


00 


Under 5% + CHTN , GA = 19.86 


FGR + PE, GA = 34.14 


same participant 
Under 5%, GA = 25.72 


Under 5% + PE, GA = 28.71 


Under 5% + PE, GA = 31.43 Under 5% + PE, G 


th Sg m 


same participant 


Fig. 3 Fractional anisotropy maps for FGR and low birthweight cohorts 
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Fig. 4 Mean FA across placenta and uterine wall as a function of gestational age 
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Fig. 5 Mean FA across placenta and uterine wall for the three cohorts, split at 30 weeks gestational 
age 


between control and growth-restricted pregnancies (Fig. 5), assessing the potential 
of FA to inform on FGR-associated placental abnormalities. 

The FA maps for all control participants (Fig. 2) show distinctive patterns in 
agreement with the literature [27]—the FA is lower in the placenta, and higher in 
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the adjacent uterine wall. These observations likely represent isotropic nature of 
maternal blood pools and fetal villous tree branching within the placenta, and the 
prevalence of anisotropic fibrous muscle tissue in the uterine wall. Another factor 
in the high FA in the uterine wall may be the coherent orientation of vasculature in 
these areas. On the other hand, the distinction between the uterine wall and placenta 
is less clear in the growth restriction placentas (Fig. 3), and many of these maps show 
areas of high FA within the placenta. 

Figure 4 plots the mean FA value within the ROI comprising the placenta and 
uterine wall. Control scans (Fig. 4, black dots) show an apparent decrease in mean 
FA over gestation, potentially reflecting microstructural changes during normal pla- 
centa maturation. On the other hand, growth restriction scans (Fig. 4, red and yellow 
dots), do not show a clear trend over gestation. At early gestational ages, control 
and compromised placentas have comparable mean FA values. Due to the downward 
FA trend in control placentas, from around 30 weeks gestation growth restriction 
placentas appear to have considerably lower FA values than control placentas. This 
difference was statisically significant, both when comparing controls with the com- 
bined FGR and low birth weight cohorts (p = 0.005, independent samples t-test), 
and when comparing controls to these two cohorts separately (Fig. 5). We tested if 
trends over GA differ between controls and growth restriction (i.e. FGR and under 
5%) cohorts by calculating a linear regression to predict mean FA based on GA, 
cohort, and the interaction between GA and cohort. The coefficients and p-values are 
given in Table 2, and Fig. 6 visualises the fits. The coefficient of the interaction term 
between GA and low birth weight is statistically significant (p = 0.002), suggesting 
a different trend over gestation. 


Table 2 Coefficients and corresponding statistics from linear regression to predict mean FA based 
on GA, cohort, and GA x cohort (interaction term) 


Coefficient Value Standard error t P > jti 
Intercept 0.8457 0.193 4.384 0.000 
Cohort (FGR) —0.4541 0.313 —1.449 0.160 
Cohort (Under5) | —0.7159 0.259 —2.765 0.011 
GA —0.0150 0.006 —2.328 0.028 
GA x Cohort 0.0188 0.011 1.769 0.089 
(FGR) 

GA x Cohort 0.0299 0.009 3.412 0.002 
(Under5) 
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Fig. 6 Linear regression to predict mean FA based on GA, cohort, and GA x cohort 


4 Discussion and Conclusion 


This study visualizes and quantifies fractional anisotropy in placentas of both control 
pregnancies and those affected by growth restriction. Encouragingly we find that the 
FA is significantly different across gestation in growth restricted pregnancies com- 
pared to controls. The fractional anisotropy is a coarse measurement that averages 
over a number of tissue properties. There are hence a number of microstructural or 
functional changes that could explain this observation. Our initial speculations of 
plausible factors behind the changes in FA are as follows. 

We observed a clear pattern of decreasing FA with gestational age for control 
placentas, suggesting that normal placental development causes a reduction in coher- 
ently orientated tissue. A consistent hypothesis is that this reduction in FA reflects 
the normal onset of terminal villi formation along the surfaces of intermediate villi, 
which occurs predominately during the third trimester [2]. On the other hand, we 
observed higher FA for placentas associated with growth restricted pregnancies after 
30 gestational weeks. This is consistent with histological findings that FGR placentas 
show a lack of side-branching terminal villi [2]. 

In this study, we utilised an MR protocol with multiple b-values and gradient 
directions. However, we only fit a relatively simple model to the data, the diffusion 
tensor. This data can support more complex models, such as anisotropic IVIM-type 
models, which separately consider the perfusion (i.e. low b-value) and diffusion 
(high b-value) signal components (e.g. [26]), and may help disentangle the tissue 
microstructure changes underlying the observed difference in FA. Another potential 
approach is to use orientation distribution functions, which might reveal differences 
in complexity of the orientation of villi (e.g. [23]). 
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There are a number of MRI-derived biomarkers that show promise for detecting 
placental dysfunction. These include: T2* relaxometry [18, 24, 25], which relates 
to oxygenation levels; structural measures [7]; apparent diffusion coefficient (ADC) 
[4, 14, 30], relating to global tissue structure; and intravoxel incoherent motion 
(IVIM) MRI perfusion fraction [8, 22, 29], which relates to blood flow. Thus far, 
dMRI-derived potential biomarkers (ADC and perfusion fraction) have considered 
tissue to be isotropic. Our results strongly suggest that the FA is also sensitive to 
placental dysfunction, and hence that quantifying tissue anisotropy is an additional 
important avenue for assessing placental health. It may be the case that multiple 
biomarkers need to be combined in order to best assess the health of an individual 
placenta. The fact that we see higher FA values for placentas with birth weight 
under the 5th percentile, as well as those diagnosed with FGR is interesting and 
merits further investigation. It is likely that a significant proportion of cases under 
the 5th percentile are undiagnosed FGR. Our results suggest that quantifying tissue 
anisotropy in the placenta could have a role to play in the detection of FGR, We will 
investigate this by combining further scanning with post-delivery placental histology 
to test the ability to distinguish FGR cases from small but otherwise healthy babies. 
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Image inpainting, 100, 101, 113 
Image registration, 255 

Image smoothing, 101, 102 
Imaging artefacts, 250 
Inclusion ROI, 156 

Inner product, 78 

Invariant, 8 

Inverse problem, 172 


Index 


ISMRM 2015 Tractography challenge, 232 
Iso-contour, 47 
Isotropy, 172 


J 
Join tree, 40, 44 


K 

Kelvin notation, 7 
Kissing tracts, 232 
Kurtosis, 213 


L 

Laplace transform, 220 

Larmor frequency, 248 

Length functional, 227, 229 

Linear diffusion, 100 

Linear homogeneous diffusion, 102 

Linearly Estimated Moments provide Orien- 
tations of Neurites And their Diffu- 
sivities Exactly (LEMONADE), 185 

Localization regime, 199 

Long-time limit, 197 

Lossless compression, 99 

Lossy compression, 99 

Lossy image compression, 104 


M 

Magnetic field, 249, 251-253 

Magnetic field gradients, 248, 249 

Magnetic field gradient waveform, 206 

Magnetic Resonance Imaging (MRD), 247, 
248, 251, 264 

Magnetic Resonance (MR), 204, 248 

Magnetic susceptibility, 248-250 

Magnetic susceptibility anisotropy, 250 

Magnetisation, 248, 249 

Magnetization density, 206 

Magnitude, 25 

Majority voting, 160 

Mean Apparent Propagator (MAP)-MRI 
method, 179 

Mean squared error (MSE), 106 

Metric, 5 

Microstructure, 194, 249, 255, 264 

Microstucture informed tractography, 152 

Mixing time, 207, 212 

Monotonous triangles, 50 

Multidimensional array, 77 

Multidimensional data, 74 
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Multi-tissue multi-shell constrained spheri- 
cal deconvolution, 251 

Multivariate data, 76 

Myelin, 249, 250 

Myelin sheath, 249, 250 

Myelin water, 250 


N 

Neumann boundary condition, 101 

Neurite Orientation Dispersion and Density 
Imaging (NODDI), 183 

Neurites, 182 

Neuronal cell, 248 

Neurons, 172 

N-mode product, 78 

Nonlinear diffusion, 100 

Nonlinear least-squares, 252 

Numerical stability, 104 


(0) 

Optimal tracts, 231 

Orientational average, 207 
Orientation distribution function, 199 


P 

Pancake, 205, 219 

Parallelization, 53 

Partial Differential Equations (PDEs), 100 
PDE-based image compression, 103 
PDE-based methods, 100 
Perona-Malik equation, 103 
Poincaré index, 40 

Pore size, 219 

Powder average, 204, 212 
Power-law, 214, 215, 217 
Pre-smoothing parameter, 112 
Processes, 173 

Progressive rank reduction, 86 
Pulesed-gradient spin-echo, 251 


Q 
Q-shell, 176 
Quadratic form, canonical, 12 


R 

Radius of gyration, 219 
Rank reduction, 81 
Receiver coil, 250 
Relaxation-anisotropy, 250 
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Restricted diffusion, 205, 219 
Riemannian metric, 227, 229 
Rose of directions, 199 

Rose of intersections, 199 


S 

Scalar product, 5 

Second-order PDEs, 113 

Short-time limit, 196 

Signal fitting, 155 

Signal representation, 173 

Signal-to-noise, 250 

Signal-to-noise ratio, 252 

Single Diffusion Encoding (SDE), 204 

Single fibre population, 251, 252 

Singular value decomposition, 76 

Somas, 173 

Spaghetti plot, 136, 137, 139 

Spatial selectivity, 85 

Specific volume, 197 

Spherical-deconvolution Informed Filtering 
of Tractograms (SIFT), 154 

Spin-spin relaxation, 248, 252 

Stability analysis, 110, 111 

Stationarity, 198 

Statistical homogeneity, 197 

Steady state, 111 

Stereology, 194 

Stick, 205, 213, 215, 218, 219 

Stochastic geometry, 194 

Streamline, 129 

Streamline clustering, 158 

Streamline loop, 157 

Streamline tractography, 227, 229 

Stress field, 57 

Structural exponent, 198 

Structural universality class, 198 

Structure tensor, 105, 114 

Subdivision, 50 

Sub-level set, 51 

Subsampling, 85 

Support function, 200 

Surfa ’lo-ice-to-volume ratio, 197 

Swelling artifact, 43 

Synthetic data, 56 
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T 

Tp-anisotropy, 250, 253, 254 

Tensor, 5, 26 

Tensor approximation, 74 

Tensor decomposition, 80 

Tensor distribution, 220 

Tensor field interpolation, 43 

Tensor norm, 79 

Tensor notation, 5 

Tensor product, 78 

Tensor rank, 79 

Tensor reconstruction, 84 

Tiltable RF coil, 247, 250, 251, 254 

Tissue microstructure, 171 

Tissue modelling, 173 

Topological invariant, 43 

Trace, 25 

Tractogram filtering, 151, 154, 160 

Tractography, 129-131, 133, 135, 139-141, 
150, 155, 226, 250, 251 

Tractometry, 250, 251, 253, 255 

Triad, 226, 228 

Trisectors, 26 

Tucker decomposition, 80 

Tucker model, 80 

Two-point load, 57 


U 

Uncertainty visualization, 125-127, 135, 
139-141 

Vv 


Vector field, 87 
Voigt notation, 7 
Von Mises stress, 57 
Vorticity field, 88 
Voxel, 204 


Ww 

Wedges, 26 

White matter, 172, 204, 218, 247-250, 252- 
254 

White Matter Tract Integrity (WMTI), 184 


